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SUMMARY 


A  media  selection  model  was  designed  and  implemented  by  Instructional 
Science  and  Development,  Inc.,  as  part  of  an  overall  automated  system  to 
facilitate  the  instructional  systems  development  (ISD)  process.  The  develop¬ 
ment  of  the  Automated  Instructional  Media  Selection  (AIMS)  model  was  based 
on  six  goals.  These  goals  were  to: 

a.  Maximize  the  use  of  pertinent  information.  This  was  accomplished 
by  automating  the  non- judgmental ,  data  manipulation  tasks  of  the  selection 
process.  The  system  accepts  media  pools  of  up  to  90  media,  and  90  instructional 
characteristics. 

b.  Provide  flexibility  in  adapting  the  model  to  specific  problems  and 
changing  technologies.  This  was  accomplished  by  allowing  user-definable  media 
pools,  and  editing  functions  which  allow  changing  and  adding  to  both  the  media 
pools  and  required  instructional  characteristics. 

c.  Provide  consideration  for  student  stage  of  learning.  The  user-definable 
aspect  of  the  iystem  allows  for  inclusion  of  any  instructional  characteristics. 

d.  Allow  for  better  selection  of  hands-on  training  devices.  Since  the 
system  is  adaptable  to  any  training  situation,  there  is  no  bias  toward 
cognitive,  learning  center  type  media. 

e .  Allow  for  variations  in  the  stage  of  front-end  analysis.  Med i a 
selections  may  be  as  general  or  as  specific  as  available  knowledge  about 
potential  instructional  characteristics  permits. 

f .  Reduce  the  time,  effort  and  costs  required  for  media  selection  procedures 
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FOREWORD 

The  Instructional  System  Development  (ISD)  model  was 
designed  to  apply  a  systems  logic  approach  to  training  program 
development  projects.  ISO  approaches  apparently  provide  superior 
training  programs,  but  they  are  costly  to  implement.  The  Human 
Factors  Laboratory  of  the  Naval  Training  Equipment  Center 
(NAVTRAEQUIPCEN)  is  undertaking  a  project  to  provide  computer 
aids  and  automation  to  some  of  the  ISD  steps  to  help  .  reduce  the 
cost  and  labor  intensiveness  of  training  program  development. 

One  effort  under  this  project  produced  a  Computer  Aided 
System  for  Developing  Aircrew  Training  (CASDAT),  a  system  for 
aiding  several  steps  of  the  ISD  Process.  The  current  report 
describes  the  development  of  an  Automated  Instructional  Media 
Selection  model  (AIMS),  which  was  designed  to  improve  upon  the 
media  .selection  component  of  CASDAT  and  other  previous  media 
selection  techniques. 

AIMS  is  more  applicable  to  a  greater  variety  of  situations 
than  previous  approaches  because  the  user  defines  and  rates 
media,  media  characteristics  and  training  objective  requirements 
to  fit  the  peculiarities  of  each  situation.  The  system  averages 
the  user's  ratings  to  indicate  the  relative  capability  of  each 
medium  in  providing  instruction  for  individual  training 
objectives.  The  ratings  also  are  tallied  across  training 
objectives,  to  provide  the  number  and  percent  of  objectives  that 
can  be  trained  with  each  medium  and  the  specific  objectives  that 
can  be  trained  with  any  combination  of  the  media.  AIMS  also 
offers  good  opportunities  for  computer  automation. 

Sample  applications  of  AIMS  are  provided  to  illustrate  use 
of  the  system  for  defining  and  selecting  both  standard, 
general-purpose  media  as  well  as  specific-function  training 
devices.  AIMS  can  be  used  "as  is"  to  aid  in  routine  efforts  to 
design,  evaluate  and  select  instructional  media.  (Early 
experience  indicates  that  a  75  percent  savings  in  time  required 
for  the  media  selection  process  can  result  from  the  use  of  AIMS.) 
Plans  are  being  made  to  implement  AIMS  in  actual  training 
analysis  and  design  applications  at  the  NAVTRAEQUIPCEN.  (CASDAT 
already  is  being  used  to  provide  solutions  to  real-world  ISD 
problems  for  the  Navy.) 

The  best  future  for  AIMS  and  media  selection  and  design  in 
general  appears  to  require  research  and  development  (R&D)  efforts 
to  define  optimal  approaches  and  data  for  application  oTAIMS  to 
a  variety  of  training  design  problems.  A  new  R&D  effort  at  this 
Laboratory--to  incorporate  computer  aids  such  as  AIMS  and  CASDAT 
into  a  more  comprehensive  system  applicable  to  the  range  of 
Marine  Corps  jobs--may  provide  progress  in  this  direction. 

Appreciation  is  expressed  to  Mr.  Joe  Puig  for  assistance  in 
reviewing  AIMS  and  this  report  and  to  Mr.  Pat  Smith  for  refining 
the  computer  programs  so  that  AIMS  could  be  reviewed. 

J  J)  S>^***4& 

ARTHUR  S.  BLAIWES,  Ph.D. 

Scientific  Officer 
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SECTION  I 


INTRODUCTION 

The  Naval  Training  Equipment  Center  (NAVTRAEQUIPCEN)  has  been  involved 
with  the  acquisition  of  aviation  weapon  systems  training  for  many  years. 

To  accomplish  such  acquisition  more  effectively,  the  NAVTRAEQUIPCEN  implemented 
an  instructional  systems  development  ( I SD)  model  which  applies  systems  logic 
to  curriculum  development.  The  cost  of  ISO  technology  can  be  justified 
through  three  methods.  The  first  of  these  is  an  intuitive  argument  that 
concerted  systematic  approaches  are  better  than  isolated  haphazard  methods. 

The  second  is  demonstrations  of  the  advantages  of  its  products.  The  third 
is  in  modifications  to  the  model  which  result  in  improvements  in  its  cost 
effectiveness.  All  three  approaches  have  been  used  in  the  past.  However, 
the  latter  is  of  major  concern  in  this  report.  A  primary  goal  of  the 
NAVTRAEQUIPCEN  is  to  reduce  the  cost  of  the  ISD  application. 

Several  efforts  toward  reducinn  ISD  costs  have  been  undertaken  recently. 

A  preliminary  s+udy  supported  by  the  MAVT°AiiniiIPCEN  identified  several  areas 
where  automated  aids  to  ISD  were  possible.  These  areas  included  everything 
from  the  analysis  of  student  entry  level  through  development  and  evaluation 
of  materials.  1  A  related  effort  supported  by  the  NAVTRAEQUIPCEN  included 
the  implementation  of  a  generic  data  base  to  be  used  in  the  ISD  phases  of 
analysis,  design,  and  development  of  Navy  aircrew  training  programs.  2  The 
present  study  was  conducted  to  assist  in  the  design,  development,  and  acquisi¬ 
tion  of  a  media  selection  component  of  the  NAVTRAEQUIPCEN  developing  prototype 
system  for  aiding  the  application  of  the  ISD  process. 

PROBLEM 

The  media  selection  component  of  ISD  has  been  a  time-consuming,  problem¬ 
atic  task  in  Navy  aircrew  training  development.  A  number  of  structured 
media  selection  models  have  been  developed  to  systematize  and  standardize 
the  media  selection  process.  However,  most  of  these  approaches  have  proven 
to  be  of  limited  value.  This  is  due  to  several  factors.  Some  are  too  broad 
or  general  and  of  little  value  for  more  advanced  or  specific  training  programs. 
Others  tend  to  overemphasize  media  for  cognitive  learning  tasks,  even  though 
a  significant  number  of  military  tasks  require  extensive  "hands-on"  training. 
Some  models  tend  to  be  overly  theoretical,  ignoring  important  practical 
considerations  such  as  cost  or  availability  of  equipment.  And  almost  all  of 
the  models  suffer  from  rigidity;  that  is,  an  inability  to  be  adapted  and 
updated  to  new  technologies,  emerging  weapon  systems,  and  changing  needs. 


Ij.  Hughes,  et  a!..  Functional  Specifications  for  Computer  Aided 
Training  Systems  Development  and  Management  (CATSDM)  Support  Functions, 
NAVTRAEQUIPCEN  Technical  Report  77-C-OOlS-l  (San  Diego,  CAT  Courseware, 

Inc. ,  July  1978). 

2 

N.  C.  Marcue,  A.  S.  Blaiwes,  A  R.  G.  Bird,  Computer  Aided  System  for 
Developing  Aircrew  Training  (CASDAT):  An  Automated  Aid  for  Developing  Aircrew 
Training,  NAVTRAEOIJIPCEN  Technical  Report  79-C-0076-1  (Arlington,  VA;  Veda, 
Inc. ,  March  1983. 
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A  variety  of  factors  contributes  to  the  difficulty  and  complexity  of  the 
media  selection  task.  First,  the  nature  of  military  jobs  poses  intrinsic 
difficulty  in  selecting  an  optimal  media  mix.  Frequently,  job  performance 
involves  a  combination  of  procedural,  perceptual,  motor,  and  knowledge 
components,  including  decision  making,  which  must  be  integrated.  Thus,  the 
learning  task  may  not  consist  of  discrete  components  which  are  easily 
associated  with  media  hardware  attributes.  Likewise,  it  is  often  difficult 
to  associate  one  specific  medium  with  the  learning  task  given  a  variety  of 
attributes  required  of  a  medium  in  order  to  best  learn  to  perform  the  task. 

It  is  often  just  as  much  the  use  of  a  medium  and  the  quality  of  the  lessonware, 
as  it  is  the  unique  attributes  of  the  media  itself,  which  determine  a  success¬ 
ful  media  application. 

In  addition,  instructional  hardware  has  evolved  co  i.ually  and  a  wide 

range  of  equipment  is  available  for  consideration.  Fur  rmore,  unbiased 

information  on  the  instructional  hardware  is  seldom  ava  ’e.  Consequently, 
there  is  little  basis  for  empirically  determining  which  4ia  hardware 
attributes  are  important  to  specific  learning  tasks. 3 

PURPOSE 

Given  the  problems  inherent  in  the  media  selection  process,  five  inter¬ 
related  goals  were  established  as  specifications  for  the  automated  media 
selection  model.  The  goals  were  to: 

a.  Provide  the  capability  to  maximize  the  use  of  information  pertinent 
to  the  media  selection  problem. 

b.  Provide  flexibility  in  modifying  the  model  to  reflect  specific 
problems  and  changing  technology  of  media  or  media  guidelines. 

c.  Provide  for  consideration  of  student  stage  of  learning. 

d.  Allow  greater,  more  detailed  selection  of  training  devices,  such  as 
simulators  or  other  hands-on  devices. 

e.  Allow  for  variations  in  the  stage  of  front-end  analysis  with  emphasis 
on  the  needs  of  planning  for  emerging  weapon  and  training  systems. 

The  purpose  of  the  effort  was  to  develop  a  working  media  selection  model 
as  an  automated  aid  for  ISD,  demonstrate  and  evaluate  the  operation  of  the 
model  in  the  NAVTRAEQUIPCEN  environment,  and  provide  plans  for  further 
development,  demonstrations  and  applications  of  the  model. 

APPROACH 

The  effort  was  organized  into  four  tasks.  First,  the  literature  on 
media  selection  was  reviewed  with  the  intent  of  drawing  the  best  features 
from  existing  procedures  and  incorporating  them  where  possible  in  an  automated 


3 

R.  Braby,  An  Evaluation  of  Ten  Techniques  for  Choosing  Instructional 
Media ,  TAEG  Report  No.  8  (Orlando,  FL:  Naval  Training  Equipment  Center, 
Training  Analysis  and  Evaluation  Group,  December  1973). 
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aid.  Second,  the  functions  of  an  automated  aid  were  selected  based  on 
potential  advantages  offered.  The  third  task  was  design,  development  and 
debugging  of  the  programs  for  the  prototype  media  selection  model  and 
development  of  a  detailed  user  guide.  The  final  task  was  application  of  the 
model  to  an  aircrew  training  problem  to  test  its  utility. 

RESULTS  AND  CONCLUSIONS 


It  was  determined  through  the  review  of  media  selection  techniques  and 
the  analysis  of  tasks  performed  during  media  selection  that  a  common  logic 
exists  in  choosing  media  that  are  effective  for  training.  Matching  the 
required  attributes  of  instruction  to  attributes  of  media  is  central  to  most 
media  models.  An  automated  instructional  media  selection  system  was  designed 
and  developed  based  on  that  logic.  The  user  is  provided  a  flexible  and 
simple  data  base  development  capability.  The  data  bast  consists  of  instruc¬ 
tional  attributes  by  which  to  analyze  information,  such  as  learning  objectives 
or  task  statements,  and  a  media  pool.  Up  to  90  media  with  90  associated 
attributes  may  be  defined  by  the  user  to  analyze  any  number  of  learning 
objectives  or  task  statements  required  by  the  user. 

Emphasis  was  placed  on  designing  the  system  to  be  a  problem  solving 
aid.  That  is,  while  existing  sets  of  media  pools  and  attributes  may  be  used, 
the  user  is  encouraged  to  create  a  problem  specific  data  base.  Each  media 
selection  problem  is  considered  uni  ;ue  until  analyzed  otherwise. 

As  illustrated  through  examples  and  demonstrations  described  later  in 
the  report,  the  system  allows  a  much  more  effective  instructional  development 
effort.  Several  areas  of  medic,  selection  which  are  not  well  attended  to  in 
some  models  are  provided.  These  include  stage  of  learning  considerations, 
detailed  selection  of  trainers,  and  variations  in  the  stage  of  front-end 
analysis.  Ultimately  such  matters  are  defined  only  through  efforts  of  the 
user,  but  the  system  can  be  ar,  efficient  tool  for  arriving  at  these  definitions 
In  addition  to  enriching  the  media  selection  process,  the  system  also  will 
save  time  and  labor.  A  small  scale  comparison  with  manual  selection  showed 
better  than  a  4:1  time  savings. 

RECOMMENDATIONS 


It  is  not  anticipated  that  any  extensive  additions  are  needed  to  the 
system.  It  should  be  recognized,  however, that  the  system  deals  only  with 

„Costs  can  dealt  with  only  on  a  relative  basis 
su  as  high,  medium,  or  low."  It  would  be  valuable  to  add  a  cos'' 
component,  several  of  which  exist,  for  the  purpose  of  detailed  cost 
comparisons  and  evaluation. 

In  addition,  the  system  offers  promise  for  other  types  of  media 
problems.  The  analysis  and  design  of  trainer  requirements,  evaluation  of 

chan^in^Hi^tpT  considered  for  acquisition,  impact  evaluation  of 
changing  media  technologies,  and  comparison  of  similar  media  are  all 
possible  areas  for  study  and  should  bo  explored. 
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SECTION  II 

REQUIREMENTS  FOR  AUTOMATED  MEDIA  SELECTION 

A  review  of  the  literature  on  media  selection  was  performed  to  determine 
the  media  selection  requirements  to  be  incorporated  into  an  automated  aid  to 
ISD.  The  five  goals,  which  were  established  as  specifications  for  the 
automated  model,  were  used  as  the  framework  for  conducting  the  review.  The 
goals  are: 

a.  Provide  the  capability  to  maximize  the  use  of  information  pertinent 
to  the  media  selection  problem. 

b.  Provide  flexibility  in  modifying  the  model  to  reflect  specific 
problems  and  changing  technology  of  media  or  media  guidelines. 

c.  Provide  for  consideration  of  student  stage  of  learning. 

d.  Allow  greater,  more  detailed  selection  of  training  devices,  such  as 
simulators  or  other  hands-on  devices. 

e.  Allow  for  variations  in  the  stage  of  front-end  analysis  with 
emphasis  on  the  needs  of  planning  for  emerging  weapon  and  training  systems. 

The  literature  review  served  three  purposes.  First,  existing  media 
selection  models  were  examined  to  identify  procedures  and  functions  which 
might  be  relevant  to  the  automated  model.  Second,  the  existing  media 
selection  models  were  analyzed  within  the  context  of  the  five  goals  and 
characteristics  (discussed  in  a  later  section)  of  an  adequate  model  in  order 
to  determine  inadequacies  of  existing  models  and  define  requirements  for 
future  models.  Third,  the  functions  to  be  included  in  the  automated  media 
selection  model  were  specified  as  a  result  of  the  review  and  critique.  The 
following  subsections  describe  the  three  aspects  of  the  literature  review. 

EXISTING  MODELS  USED  IN  NAVY  AIRCREW  TRAINING 

A  sample  of  representative  Navy  aircrew  training  objectives  was  used 
to  assess  the  procedures  and  functions  of  four  media  selection  models  which 
have  been  used  in  Navy  aircrew  training  systems.  The  models  are:  (1)  the 
Training  Effectiveness  and  Cost  Effectiveness  Prediction  (TECEP)  model  as 
originally  developed  by  the  Navy  Training  Analysis  and  Evaluation  Group 
(TAEG),  ^  (2)  the  Training  Effecti veness-Cost  Effectiveness  Prediction 
(TECEP)  technique  as  it  was  enhanced  by  TAEG,  5  (3)  the  MIL-T  model 


^R.  Braby,  et  al.,  Staff  Study  on  Cost  and  Training  Effectiveness  of 
Proposed  Training  Systems,  TAEG  Report  No.  1  (Orlando,  FL:  Naval  Training 
Equipment  Center,  Training  Analysis  and  Evaluation  Group,  1972). 

5R.  Braby  et  al.,  A  Technique  for  Choosing  Cost  Effective  Instructional 
Delivery  Systems,  TAEG  Report  No.  16  (Orlando,  FL:  Naval  Training  Equipment 
Center,  Training  Analysis  and  Evaluation  Group,  April  1975). 
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described  in  the  Military  Specification:  Training  Requirements  for  Aviation 
Weapon  Systems  (MIL- '*  29053)  6  and  (4)  the  LAMPS  MARK  III  methods/media  model 
developed  by  IBM.  7  To  simplify  discussion  throughout  this  report,  the 
original  TECEP  model  is  referred  to  as  the  TAEG  model  and  the  enhanced 
model  is  identified  as  the  TECEP  technique. 

Each  of  the  models  performs  the  following  five  tasks  with  varying 
degrees  of  detail  found  in  the  procedures: 

a.  Identify  specific  objectives  and  training  tasks 

b.  Analyze  each  specific  objective  by  attribute 

c.  Match  instructional  attributes  against  defined  media  pool  attributes 

d.  Rank  media  for  training  effectiveness 

e.  Perform  costs/constraints  trade-offs  for  final  selection. 

Since  cost  effectiveness  considerations  were  beyond  the  scope  of  the 
effort,  the  models  were  evaluated  to  the  point  of  rank-ordering  the  media 
for  training  effectiveness.  The  requirements  of  the  last  task,  final 
selection  based  on  trade-offs,  were  only  reviewed  briefly.  The  procedures 
for  each  model  are  summarized  below. 

TAEG  MODEL.  The  TAEG  model  was  developed  to  apply  the  concepts  of  learning 
theory,  economic  analysis,  and  other  information  acquired  by  use  of  the 
scientific  method  to  the  task  of  training  system  design.  The  model  consists 
of  nine  steps.  However,  the  last  five  steps  relate  to  selecting  media  on 
the  basis  of  cost  effectiveness,  rather  than  training  effectiveness. and  they 
are  not  discussed.  During  these  steps  the  media  alternatives  are  rank- 
ordered  and  trade-offs  are  performed  for  final  selection.  A  description  of 
the  four  training  effectiveness  steps  is  provided  below. 

a.  Task  Description  and  Analysis.  Each  task  to  be  performed  is  described 
in  terms  of  the  skins  and  knowledges  required  to  accomplish  the  functions. 
Where  possible,  specific  training  objectives  should  be  provided  as  the  task 
descriptions. 

b.  Personnel  Characteristics.  A  definition  of  student  entry  level 
character! sties  is  used  to  determine  the  starting  point  for  training. 


6  Military  Specification:  Training  Requirements  for  Aviation  Weapon 
Systems,  MIL-T-29053  (Orlando,  FL:  Naval  Training  Equipment  Center,  October  1977). 

7  LAMPS  MARK  III  Methods/Media  Selection  Model  (Manassas,  VA:  IBM 
Federal  System  Division,  1979). 
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c.  Training  Tasks  and  Training  Stages.  Training  tasks  are  organized 
into  a  series  of  stages  that  progress  through  prerequisite  skills  to  advanced 
skills.  The  general  characteristics  of  the  tasks  to  be  accomplished  within 
each  stage  are  identified  as  one  of  the  following  13  task  categories: 

(1)  Recalling  Facts  and  Principles 

(2)  Recalling  Procedures 

(3)  Non-Verbal  Identification 

(4)  Non-Verbal  Detection 

(5)  Using  Principles,  Interpreting,  Inferring 

(6)  Making  Decisions 

(7)  Continuous  Movement 

(8)  Verbal  Detection  and  Identification 

(9)  Positioning  and  Serial  Movement 

(10)  Repetitive  Movement 

(11)  Written  Verbalization 

(12)  Oral  Verbalization 

(13)  Other  Verbalization,  including  Signs. 

d.  Determination  of  Useful  Media  Options.  The  training  system 
designer  uses  the  media  selection  matrix  to  identify  a  cluster  of  media 
which  satisfy  the  characteristics  of  specific  training  tasks.  A  sample  page 
from  the  media  selection  matrix  for  the  task  category  Non-Verbal  Detection 

is  provided  in  Figure  1.  As  shown,  the  first  column,  Task  Categories,  contains 
descriptions  of  the  common  behavioral  attributes  and  examples  for  the  task 
categories  Incorporated  into  the  matrix.  The  next  column.  Task  Elements, 
divides  each  task  category  into  elements  so  learning  guidelines  can  be 
associated  with  four  standard  elements  of  a  task  (stimulus,  response,  feedback, 
personal  environment).  The  column  Learning  Guidelines,  contains  rules-of- 
thumb,  based  on  learning  theory,  for  designing  and  using  a  training  system  for 
a  specific  type  of  task.  A  total  of  105  learning  guidelines  are  identified 
and  associated  with  the  13  task  categories.  The  column.  Implications  for 
Media  Selection,  contains  one  or  more  implications  associated  with  each 
learning  guideline.  They  serve  to  simplify  the  task  of  determining  if  a 
specific  medium  facilitates  the  use  of  the  learning  guideline.  The  media 
pool  consists  of  20  options  which  are  defined  in  a  separate  table.  The  "M" 
and  "F"  designations  are  described  as  notes  at  the  bottom  of  the  matrix.  8 


8 


See  footnote  4  on  page  10. 


12 


Figure  1.  A  sample  page  from  the  TAFG  media  selection  matrix. 


NAVTRAEQUIPCEN  79-C-0 104-1 


TECEP  TECHNIQUE.  The  TECEP  technique  was  designed  to  provide:  (1)  a  set  of 
learning  guidelines  appropriate  to  Navy  job  tasks,  and  (2)  a  method  for 
selecting  cost  effective  instructional  delivery  systems  that  support  the  use 
of  those  learning  guidelines.  Aagard  and  Braby  report  that  the  technique 
uses  learning  algorithms  to  combine  and  sequence  the  guidelines.  9  As 
defined  by  Braby,  et  al.,  ".  .  .a  learning  algorithm  is  a  step-by-step 
prescription  for  a  student  to  follow  in  learning  any  specific  task  in  a 
class  of  learning  tasks.  .  ."  10 

Use  of  the  TECEP  technique  requires  expertise,  and  is  perhaps  best  viewed 
as  a  job  aid  for  an  experienced  training  system  designer.  The  TECEP  technique 
is  a  three-step  approach  to  media  selection.  The  first  step  presents  a  set 
of  learning  guidelines  and  an  algorithm  for  each  class  of  training  tasks. 

The  second  step  identifies  instructional  delivery  systems  capable  of  supporting 
each  set  of  learning  guidelines  and  algorithm,  while  the  third  step  provides 
a  cost  model  for  projecting  the  costs  of  each  instructional  delivery  system 
alternative  in  a  specific  training  setting.  Only  steps  one  and  two  are 
discussed  below,  since  cost  effectiveness  considerations  were  beyond  the 
scope  of  the  effort. 

The  initial  step  is  to  classify  and  group  the  training  objectives  according 
to  the  type  of  learning  algorithm  required  to  accomplish  the  objective.  This 
requires  matching  each  training  objective  in  the  proposed  training  system  with 
the  learning  algorithms  for  achieving  the  objective.  A  table  describing  the 
types  of  learning  algorithms  and  the  characteristics  (action  verbs,  behavioral 
attributes,  and  examples)  of  the  training  objectives  that  they  support  is  used 
to  compare  against  the  characteristics  of  the  objectives.  The  12  types  of 
learning  algorithms  are: 

a.  Recalling  bodies  of  knowledge 

b.  Using  verbal  information 

c.  Rule  learning  and  using 

d.  Decision  making 

e.  Detecting 

f.  Classifying 

g.  Identifying  symbols 

h.  Voice  comunicating 


9j.  A.  Aagard  and  R.  Braby,  Learning  Guidelines  and  Algorithms  for 
Types  of  Training  Objectives,  TAEG  Report  No.  2$  (Orlando,  FL:  Naval 
Training  Equipment  Center,  Training  Analysis  and  Evaluation  Group,  March 
1976). 

10 See  footnote  5  on  page  10. 
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i.  Recalling  procedures  and  positioning  movement 

j.  Steering  and  guiding;  continuous  movement 

k.  Performing  gross  motor  skills 

l.  Attitude  learning 

The  second  step  is  to  identify  instructional  delivery  systems  that 
support  the  use  of  the  learning  algorithm  required  for  each  training  objective. 
This  step  actually  consists  of  two  substeps:  (1)  identification  of 
instructional  delivery  systems  {media  options),  and  (2)  practicality  tests. 

In  the  first  substep,  all  of  the  instructional  delivery  systems  with 
stimulus,  response,  and  feedback  capabilities  required  to  support  the  events 
in  the  learning  algorithm  are  identified.  Instructional  Delivery  System 
Selection  Charts  developed  for  each  of  the  12  learning  algorithms  serve  as 
job  aids  for  training  system  designers.  As  shown  in  the  example  in  Figure  2, 
a  list  of  instructional  delivery  systems  is  provided  across  the  top  of  the 
chart,  while  special  criteria  are  listed  down  the  left  side.  An  "X"  is 
placed  in  the  cells  of  the  systems  that  meet  the  special  criteria.  The 
training  system  designer  can  easily  identify  candidate  delivery  systems  by 
checking  the  relevant  selection  criteria  on  the  left  side  and  looking  for 
the  delivery  systems  that  have  an  "X"  in  that  cell. 

An  alternate  approach  can  be  used  to  consider  instructional  delivery 
systems  not  included  in  the  charts.  Two  job  aids  are  provided  for  experienced 
training  system  designers:  (1)  a  list  of  55  generic  media  characteristics 
which  includes  stimulus,  response,  and  feedback  characteristics  that  can  be 
used  to  describe  types  of  instructional  media,  and  (2)  a  list  describing  89 
general  types  of  media  that  can  be  incorporated  into  instructional  delivery 
systems.  With  these,  the  generic  media  characteristics  required  to  implement 
the  learning  algorithms  can  be  specified  and  the  media  containing  these 
characteristics  can  be  identified.  This  approach  requires  expert  knowledge 
of  media,  the  algorithms,  and  the  subject  matter. 

The  second  substep  in  step  two  is  testing  the  candidate  instructional 
delivery  systems  for  practicality  and  eliminating  impractical  delivery  system 
candidates  prior  to  cost  estimating  in  the  third  step.  Each  delivery 
system  is  evaluated  in  terms  of  the  following  11  criteria: 

a.  Marginal  technical  solution 

b.  State  of  the  art  system 

c.  Size  of  the  system 

d.  Interface  with  existing  programs 

e.  Lead  time  required  to  produce  system 

f.  Budget  cycle  constraints 
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INSTRUCTIONAL  DELIVERY  SYSTEM  CHART  FOR  THE  ALGORITHM 

RECALLING  BODIES  OF  KNOWLEDGE 


Alternative  Inet 


To  choose  a  delivery 
systemi 

1.  Place  a  V"  Ulght 
pencil)  in  boxes 
representing  criteria 
(rows)  that  must  be  set 

2.  Select  the  delivery 
systeas  (columns)  that 
have  an  “x“  in  each  row 
designated  by  a  V** 
These  are  the  candidate 

l  delivery  system 


Delivery  Approaches 
permitting  the  Application 
of  All  Learning  Guidelines 
and  Algorithm 


Delivery  Approaches  NOT 
Permitting  Complete 
Application  of  Learning 
Guidelines  end  Algoritnr 


Criteria  for 
Selecting 
Instructional 
Delivery 

Systems 


Stimulus  Criteria 

• 

visual 

Limited 

rull 

• 

Visual 

Spectrum 

Full  Color 

• 

Audio 

Voice  Sound 

Range 

Full  Sound 

Bangs 

Training  Setting  Criteria 

%  individual  Trainees  st  Fixed  Location 

•  Individual  Trainees  with  Simultaneous 
Instruction  at  Many  Location* 

•  Individual  Trainaes  with  Independent 
Instruction  at  Any  Location 

%  Small  Group 

•  Largs  Group  at  a  Singie  Location 

•  Team  Setting 


Administrative  Criteria 

•  Site  of  Courseware  and  Special  Hard¬ 
ware  Development 

Local 

Central 

•  Magnitude  of  Acquisition  Coat 

Low 


Figure  2.  A  sample  selection  worksheet  from  the  TECEP  technique. 
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g.  User  acceptance  of  innovations 

h.  Resources  for  courseware  development 

i.  High  cost  system  alternative 

j.  Learning  style  of  trainees 

k.  Any  other  applicable  constraints.^ 

MIL-T  MODEL.  The  media  selection  model  described  in  MIL-T-29053  uses  a 
sorting  algorithm  to  divide  objectives  into  categories  and  then  select  media 
alternatives  from  the  following  media  pool: 

a.  Workbook 

b.  Mediated  Interactive  Lecture 

c.  Slide  tape 

d.  Videotape 

e.  Random  Access  Slide 

f.  CAI 

g.  CAI  Simulation 

h.  Simulation 

i.  Actual  Equipment 

Figure  3  depicts  the  selection  algorithm.  The  decision  points  (Q1-Q5) 
refer  to  questions  1  through  5  described  in  Figure  4.  These  questions  attempt 
to  determine  the  critical  features  of  an  objective  which  determine  the  best 
media  for  teaching  the  objective.  Question  1  is  concerned  with  the  type  of 
learning  involved  and  Question  2  focuses  on  the  level  of  content.  Question 
3  is  concerned  with  variability  in  the  examples  and  practice  items  necessary 
for  accomplishing  an  objective.  Question  4  focuses  directly  on  the  minimal 
media  attributes  necessary  to  teach  an  objective,  while  Question  5  addresses 
the  memory  load  involved.  To  use  the  model,  one  enters  the  selection  algorithm 
at  the  box  labeled  Q1  in  the  center  of  the  algorithm.  Then,  one  proceeds 
through  the  decision  points  answering  the  questions  until  a  terminal  point 
is  reached.  The  terminal  points  (M1-M44)  refer  to  three  or  four  media 
selected  for  each  training  objective  from  the  pool  listed  in  rank  order  accord¬ 
ing  to  their  desirability  for  an  objective,  or  provide  additional  information 
or  questions  to  be  asked  in  the  selection  process. *2 


H See  footnote  5  on  page  10. 
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See  footnote  6  on  page  11. 
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Figure  3.  Algorithm  for  the  MIL-T  media  selection  model. 
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Q1  is  the  number  of  the  choice  which  best  answers  this 
question  for  a  given  objective: 

What  is  the  level  of  behavior  expected  of  the  student 
in  this  segment? 

1  a  familiarization 

2  *  discriminated  recall 

3  ■  rule  using 

Q2  is  the  number  of  the  choice  which  best  answers  this 
question  for  a  given  objective: 

What  level  of  content  is  being  taught  in  this  segment? 

1  *  familiarization 

2  *  paired  associate 

3  *  concept 

4  =  rule 

Q3  is  the  number  of  the  choice  which  best  answers  this 
question  for  a  given  objective: 

Is  the  minimum  critical  set  of  instances  the  student 
needs  to  see  small  or  large? 

1  ■  small 

2  ■  large 

Q4  is  the  number  of  the  choice  which  best  answers  this 
question  for  a  given  objective: 

What  is  the  minimum  display  requirement? 

1  =  verbal  and/or  symbolic  and/or  static  simple 

pictorial 

2  *  verbal  and/or  symbolic  and/or  static  complex 

pictorial 

3  =  dynamic  pictorial 

4  =  interactive 

Q5  is  the  number  of  the  choice  which  best  answers^ this 
question  for  a  given  objective: 

If  the  memorization  component  of  this  objective  large 
or  small? 

1  ■  small 

2  •  large 


Figure  4.  Key  to  the  meaning  of  Q1-Q5  decision  points 
in  the  MIL-T  selection  model. 
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LAMPS  MARK  III  MODEL  .  The  LAMPS  MARK  III  model  was  designed  to  provide  a 
systematic  analysis  of  the  methods/media  requirements  for  each  objective  in 
the  LAMPS  MARK  III  training  program.  The  model  emphasizes  three  major  factors 
in  prioritizing  objectives  and  selecting  media: 

a.  "Hands-on"  versus  "cognitive"  requirements 

b.  Instructional  characteristics  required 

c.  Level  and  type  of  learning  to  be  achieved. 

The  model  assumes  a  subject  matter  expert  and  an  instructional  technologist 
work  together  to  make  judgmental  decisions  about  these  three  factors.  The 
technique  requires  going  through  the  following  five-step  process  for  each 
objective  using  the  sample  worksheet  shown  in  Figure  5. 

a.  Determine  it  objective  Is  hands-on  or  cognitive.  An  obioctive  is 
"hands-on"  if  it  requires  the  use  of  operational  equipment  or  simulations 
to  learn  the  specified  behavior.  Any  objective  which  can  be  fulfilled 
through  verbal,  pictorial,  symbolic,  or  any  other  non-operational  means 

is  classified  as  "cognitive." 

b.  Determine  instructional  characteristics.  The  user  determines  the 
instructional  characteristics,  or  media  attributes,  required  to  accomplish 
an  objective  effectively  and  efficiently  by  checking  off  the  essential 
requirements  on  the  worksheet. 

c.  Select  appropriate  methods/media.  Once  the  instructional  character¬ 
istics  are  checked  off  on  the  worksheet,  they  are  matched  against  the 
capabilities  of  each  alternative  in  the  media  pool.  This  is  accomplished  by 
use  of  separate  media-by-characteristics  matrices  for  hands-on  and 
cognitive  media. 

d.  Determine  learning  classification.  At  this  point,  the  selected 
media  are  identified  on  the  worksheet  with  no  ranking.  Objectives  are  then 
classified  by  the  assessed  level  of  learning.  The  learning  level  categories 
are  shown  under  the  columns  for  hands-on  and  cognitive  tasks  on  the  left 
side  of  Figure  5.  For  hands  on  tasks,  the  fidelity  requirements  of  the 
media  used  are  likely  to  increase  with  each  level  of  learning.  For  cognitive 
skills,  masteryis  gained  through  a  progression  of  learning  experiences. 

+ 

e.  Prioritize  selected  media.  The  level  of  learning  is  used  to 
prioritize  the  methods/media  selections.  Priority  assignment  tables  (one 
for  cognitive  and  one  for  hands-on)  rank  order  the  media  in  the  pool  by 
each  level  of  learning.  Using  the  priority  assignment  tables  the  selected 
media  are  ranked  by  placing  the  appropriate  number  next  to  the  media  on  the 
worksheet.  13 


13 


See  footnote  7  on  page  11. 
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CRITIQUE  OF  EXISTING  MODELS 

The  four  media  models  --  TAEG,  TECEP,  MIL-T,  and  LAMPS  MARK  III  --  were 
evaluated  by  the  study  team  within  the  context  of  the  five  goals  previously 
discussed  for  the  automated  aid  and  seven  characteristics  (see  Table  1) 
specified  for  an  adequate  model  in  order  to  identify  existing  inadequacies 
and  determine  requirements.  The  characteri sties  of  an  adequate  media 
selection  model  were  defined  in  a  study  conducted  by  TAEG  to  compare  and 
evaluate  ten  techniques  for  selecting  instructional  media.  ^  Nine  detailed 
characteristics  were  recommended  as  a  set  of  guidelines  to  be  used  to 
improve  and  expand  existing  models.  Seven  of  these  characteristics,  which 
were  considered  relevant  to  the  present  effort,  are  summarized  in  Table  1. 

Starting  with  the  first  characteristic  in  Table  1,  it  was  judged  that 
the  four  existing  models  were  techniques  which  assisted  professionals  in 
handling  media  selection  with  varying  degrees  of  adequacy.  In  terms  of  the 
last  characteristic,  several  of  the  models  were  previously  tested  with 
objectives  from  several  projects  and  none  of  them  was  considered  to  be  widely 
applicable.  Each  model  was  found  to  be  limited  and  inflexible  in  one  or  all 
of  the  remaining  five  characteri sties  summarized  in  Table  1.  The  following 
paragraphs  describe  each  model. 

Experience  with  the  MIL-T  model  indicated  that  it  was  adequate  for 
grouping  objectives  into  general  categories  of  media  requirements,  but  there 
were  problems  and  limitations.  First,  the  limited  media  pool  kept  the 
selections  at  a  broad,  general  level.  There  was  no  provision  for  making 
finer  comparisons,  such  as  among  types  of  CAI  or  types  of  simulators. 

Further,  it  seemed  difficult  to  add  additional  media  to  the  pool,  since  the 
logic  governing  the  relationships  amona  the  instructional  characteristics 
and  the  media  themselves  was  not  explicit. 

A  second  problem  with  the  MIL-T  model  was  its  emphasis  on  "cognitive," 
learning  center  type  media.  Workbook  and  mediated  interactive  lecture  each 
appear  in  27  of  the  44  terminal  points.  Objectives  considered  to  be  "hands- 
on"  were  generally  lumped  into  one  broad  "simulator"  category,  which  requires 
another  selection  process  (not  provided  by  the  model)  to  be  further  defined. 

In  fact,  only  three  terminal  points  present  simulator  as  an  alternative. 

A  third  difficulty  was  the  assumption  that  the  user  was  knowledgeable 
in  esoteric  instructional  technology.  Phrases  such  as  "di scriminated- 
recall  level  of  behavior,"  "paired-associate  level  of  content"  anc}  "static 
complex  pictorial"  are  likely  to  have  little  meaning  to  a  person  with  strictly 
subject  matter  expertise  and  general  training  experience  within  a  given  area. 
Yet  that  same  person  might  be  the  one  best  suited  to  clearly  define  the 
essential  characteristics  required  of  the  media.  Even  though  the  model  is 
automated  in  the  CASDAT  implementation  of  the  system,  its  usefulness  was 
1 imi ted. 


^  See  footnote  3  on  page  £. 

15  „ 

See  footnote  2  on  page  7. 
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TABLE  1.  SEVEN  CHARACTERISTICS  OF  AN  ADEQUATE  MEDIA  SELECTION  MODEL 

_ Character!  stic _ Description 

1.  Technique  for  Professionals  Tool  to  assist  instructional  technolo- 


2.  Useful  Variables 


3.  Media  Alternatives 


4.  Training  System  Descriptions 


5.  Growth  Potential 


6.  Data  Manipulation 


7.  Testing 


gists  in  performing  a  judgmental  task 
that  cannot  be  fully  procedural i zed . 
Formal  process  with  a  set  of  operation¬ 
ally  defined  terms  and  logic  based  on 
theoretical  concepts  about  the  processes 
of  teaching  and  learning. 

Two  types:  (1)  variables  that  describe 
a  training  program  as  a  learning  system 
(i.e.,  stimulus  modes,  response  modes, 
feedback  modes,  learning  strategies,  arc 
learning  styles.)  and  (2)  variables  that 
define  costs/constraints  trade-offs. 

Extensive  pool,  including  both  tradi¬ 
tional  and  new  forms  of  instructional 
media,  and  media  for  supporting  classroom 
and  individualized  instruction,  on-the- 
job  training. 

Descriptions  of:  (I)  broad  instructional 
strategies  for  general  types  of  learning 
tasks,  (2)  specific  mixes  of  media  for 
specific  learning  tasks,  and  (3)  specific 
design  and  use  parameters  to  be  incor¬ 
porated  into  training  system  specifica¬ 
tions  for  production  or  procurement  of 
media  packages. 

Capability  for  adding  or  changing  media 
types,  learning  guidelines,  and  economic 
factors.  Applicable  in  a  broad  variety 
of  current  and  emerging  training 
si tuations. 

Emphasis  on  decision  making,  with  data 
retrieval  and  processing  as  a  minor  task. 
Manual  computations  and  mechanics  of 
combining  factors  through  the  use  of 
automatic  data  processing,  two  dimension 
tables,  and  simple  math  models. 

Tested  with  objectives  from  several 
training  projects.  Adaptable  to  a 
variety  of  training  projects. 
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Apo'-ication  of  the  TECEP  technique  proved  to  be  a  laborious  task. 

Several  data  retrieval  and  matrix  matching  tasks  appeared  to  be  well  suited 
to  automation  to  reduce  labor  intensiveness.  In  general,  the  model  was  not 
temple  to  user  requirements.  No  technique  for  adding  media  or  instructional 
attributes  to  the  Instructional  Systems  Delivery  Charts  was  provided  and  use 
c*  the  a’ternative  approach  was  difficult  and  time-consuming  although  the 
media  pool  is  extensive.  Variations  in  the  level  of  descriptions  of  media 
a'  -'•natives  were  not  included.  The  media  selections  were  quite  specific, 

'•at r t-r  than  broad  or  overly  general.  The  model  was  not  adaptable  to 
variations  in  the  stage  of  development  for  proposed  training  programs. 

In  contrast,  the  TAEG  model  provided  more  user  flexibility.  The 
format  of  the  media  selection  matrix  made  it  fairly  easy  to  use  because  the 
task  categories,  task  elements,  learning  guidelines  and  media  pool  were 
contained  in  one  matrix.  The  existing  media  pool  was  limited  and  the 
selections  were  specific,  but  it  was  stated  that  additional  media  options 
could  be  added. 

Exoerience  with  the  LAMPS  MARK  III  model  was  generally  a  positive  one. 

"he  worksheet  approach,  where  the  user  simply  checks  off  the  appropriate 
instructional  characteristics,  has  several  advantages.  First,  it  was  simple 
and  straightforward,  requiring  little  interpretation  of  esoteric  terminology. 
Second,  it  allowed  the  user  to  focus  on  the  single  task  of  matching  objectives 
to  instructional  attributes.  This  made  it  possible  to  complete  the  matching 
task  quickly  and  with  a  minimum  of  fatigue.  The  process  of  going  through 
the  media  pool  matrices  and  selecting  the  media  with  the  required  instructional 
characteri sties  was  time  consuming  and  mechanical.  These  characteristics 
made  this  process  a  prime  candidate  for  automation. 

The  media  pool  options  were  broad  and  general,  but  probably  suited  to 
the  requirements  of  the  LAMPS  MARK  III  training  program.  A  comprehensive, 
but  generalized,  set  of  cognitive  media  is  included  in  the  media  pool.  For 
any  given  cognitive  medium  listed  in  the  pool,  a  number  of  devices,  methods, 
and  techniques  can  be  employed  for  actual  implementation.  Where  minimal 
i nstructional  data  are  available  for  making  selections,  cost  and  practicality 
most  influence  the  media  to  be  selected.  The  model  selects  generalized  "hands- 
on"  devices,  primarily  on  the  basis  of  fidelity  requirements  indicated  by  the 
level  of  learning.  The  design  of  these  devices  is  assumed  to  take  place  at  a 
later  date  based  on  the  instructional  objectives.  No  capability  is  provided, 
however,  for  assessing  the  value  of  specific  device  parameters  such  as  stated 
in  characteristic  4  (third  part)  of  Table  1. 

The  relationship  between  the  characteristics  identified  as  inadequate 
in  the  existing  models  and  the  goals  established  for  the  automated  aid  are 
described  below.  To  facilitate  the  discussion,  the  five  goals  are  summarized 
as: 


a.  Maximize  the  use  of  pertinent  information  (Goal  1). 

b.  Provide  flexibility  to  modify  the  model  to  reflect  specific 
problems  and  changing  technology  (Goal  2). 
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c.  Consider  student  stage  of  learning  (Goal  3). 

d.  Allow  more  detailed  selection  of  training  devices  (Goal  4). 

e.  Allow  for  variations  in  the  stage  of  front-end  analysis  (Goal  5). 

Starting  with  characteristic  two  in  Table  1,  each  of  the  models  provides 
a  predetermined  set  of  variables  to  be  used  in  analyzing  a  given  media 
selection  problem.  The  usefulness  of  these  variables  changes  considerably 
from  one  training  situation  to  another.  The  more  flexibility  the  media 
selection  team  has  in  defining  the  useful  variables,  the  better  the  model  will 
address  the  given  media  selection  problem.  Therefore,  the  inadequacies  found 
in  defining  useful  variables  relate  to  the  requirements  identified  in  all  of 
the  goals. 

All  of  the  models  were  considered  inadequate  on  characteristic  three, 
media  alternatives.  Each  model  provided  a  limited  number  of  options  in  the 
media  pool  and  the  descriotions  were  broad  and  general.  Although  the  TECEP 
technique  has  an  alternative  approach  with  an  extensive  media  pool  to  select 
from,  it  was  too  time-consuming  and  tedious  to  be  considered  feasible  for 
large  training  programs.  These  inadequacies  support  the  requirements  stated 
in  Goals  1,  2,  and  4. 

None  of  the  models  satisfied  the  requirements  of  characteristic  four, 
training  system  descriptions.  The  models  provide  descriptions  of  either 
broad  i nstructional  strategies  fo<-  general  tasks  or  specific  media  mixes  for 
specific  tasks.  They  do  not  permit  detailed  descriptions  which  could  be 
incorporated  into  training  system  specifications  for  media  packages.  The 
capability  to  address  different  types  of  training  system  descriptions  is 
required  if  the  technique  is  to  be  applicable  to  a  variety  of  media  selection 
problems.  This  inadequacy  relates  to  the  requirements  of  Goals  1,  2.  4 
and  5. 

Each  of  the  models  was  deficient  in  terms  of  characteristic  five, 
growth  potential.  There  was  limited  capability  for  adding  to  or  changing 
the  parameters  of  any  of  the  models.  Each  model  offers  a  predefined  approacn 
for  analyzing  a  given  training  situation.  The  issue  is  one  of  which  media 
selection  model  to  use,  not  how  to  change  the  model  parameters.  The 
capability  for  adaptation  supports  all  five  of  the  goals. 

Characteristic  six,  data  manipulation,  supports  the  concept  of  an 
automated  aid  to  media  selection  for  the  ISD  process.  The  inadequacies 
described  in  existing  models  were  used  to  derive  the  requirements  outlined 
below. 
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FUNCTIONS  OF  AUTOMATED  MEDIA  SELECTION 

The  functions  to  be  incorporated  into  the  automated  aid  to  ISD  were 
derived  from  the  literature  review  on  the  current  status  of  media  selection 
and  the  evaluation  of  existing  models  used  in  Navy  aircrew  training.  As 
described  earlier,  the  typical  media  selection  model  accomplishes  five  tasks: 
(1)  initially,  a  set  of  specific  objectives  and  tasks  is  required;  (2)  these 
objectives  and  tasks  are  analyzed  against  a  predetermined  set  of  instructional 
character? stics/attributes;  (3)  the  identified  attributes  are  then  matched 
against  the  attributes  of  the  media  in  a  predefined  pool  to  select  media 
al ternati ves ;  (4)  the  alternatives  are  rank-ordered  for  training  effective¬ 
ness;  and  (5)  the  final  media  selection  is  based  on  costs/constraints  trade¬ 
offs. 

An  automated  aid  can  perform  these  tasks  more  efficiently,  as  well  as 
provide  the  user  team  of  instructional  technologists,  media  specialists,  and 
subject  matter  experts  with  expanded  capabilities  and  flexibility  to  adapt 
the  process  to  the  requirements  of  the  media  selection  problem  at  hand.  The 
scope  of  the  expanded  capabilities  and  flexibility  requirements  were  based  on 
the  inadequacies  described  for  existing  models.  The  emphasis  of  the  automated 
aid  is  or.  decision  making  with  appropriate  personnel  performing  judgnental 
tasks  and  the  computer  accomplishing  tedious  mechanical  processing,  matching, 
and  retrieval  tasks.  The  growth  potential  of  the  automated  aid  includes  the 
capability  to  define  an  extensive  media  pool  and  provide  variable  descriptions 
(broad  instructional  strategies,  or  specific  media  mixes,  or  training  system 
specifications)  based  on  the  information  available  in  current  and  emerging 
training  situations.  Figure  6  presents  a  conceptual  flow  of  the  tasks  in 
the  expanded  media  selection  process.  To  facilitate  explanation  of  the 
expanded  capabilities,  the  tasks  shown  in  Figure  6  have  been  organized  into 
the  following  five  functions: 

a.  Problem  analysis 

b.  Definition 

c.  Information  analysis 

d.  Matching 

e.  Trade-offs. 

Beginning  at  the  top  of  Figure  6,  the  first  set  of  tasks  in  the  automated 
model  requires  the  user  team  to  perform  a  problem  analysis.  First,  they  must 
determine  whether  any  existing  model  meets  the  needs  of  this  particular  media 
selection  problem.  A  variety  of  differences  seems  to  occur  in  media  selection 
problems:  for  example,  emphasis  on  particular  types  of  trainers  which  may  be 
unique  to  the  situation,  differences  in  how  one  must  account  for  entry  level 
characteristics  (such  as  reading  problems  or  other  learning  handicaps), 
training  which  requires  extensive  on-the-job  training  or  high  performance- 
based  instruction.  If  an  existing  model  does  not  meet  the  needs,  then  there 
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Figure  6.  Flow  of  tasks  in  automated  media  selection 
with  automated  aids. 
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Figure  6.  Continued. 
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are  two  general  reasons:  (1)  the  attributes  which  must  be  used  to  describe 
the  instruction  are  inadequate,  and  (2)  the  media  pool,  which  is  to  be 
selected  from, is  inadequate.  Therefore,  the  problem  analysis  includes  asking 
questions  concerning  these  two  aspects  of  media  selection.  If  it  is  determined 
that  different  attributes  or  media  are  required,  then  the  tasks  in  the 
definition  phase  are  performed.  These  tasks  are  identified  by  a  "2"  in  the 
flowchart.  The  user  team  identifies  the  relevant  attributes  for  analysis  of 
the  instructional  requirements,  determines  the  media  to  be  considered,  and 
makes  judgments  on  the  media  capabilities  by  defined  attributes. 

The  user  team  now  enters  the  information  analysis  phase.  In  this  phase, 
the  information  available  to  describe  the  training  requirements  is  analyzed  to 
determine  which  attributes  and  perhaps  how  much  capability  the  training  system 
will  require.  Ordinarily,  this  is  done  on  the  basis  of  specific  statements  of 
objectives  and  tasks  which  are  required  to  be  performed.  However,  such 
specific  information  is  not  always  available.  Three  additional  forms  of 
information  are  possible:  (1)  generic  objectives  and  tasks,  (2)  equipment 
and  personnel  descriptions,  and  (3)  mission  descriptions.  These  are  shown 
as  "B,"  "C,"  and  "D"  in  Figure  6.  The  tasks  designated  as  "A"  within  the 
dotted  line  represent  those  found  in  typical  media  selection  models. 

If  specific  objectives  and  task  statements  exist,  or  time  is  available 
to  determine  them,  then  each  objective  may  be  analyzed  by  the  attributes 
required  for  training  and  matched  against  the  media  pool.  A  different 
approach  must  be  taken  for  the  other  three  forms  of  information.  If  the 
training  system  analysis  for  an  aircraft  is  not  complete,  CASDAT  is 
intended  to  provide  generic  tasks  and  objectives  for  any  aircraft.  The 
data  base  may  be  searched  to  determine  past  media  selections  for  the  generic 
objectives  or  the  objectives  reanalyzed  for  a  newly  specified  media  pool,  if 
the  past  media  selections  appear  to  fit  the  new  training  situation,  then  the 
matching  and  trade-off  phases  may  proceed. 

However,  trade-offs  and  constraints  often  differ  from  one  media 
selection  situation  to  another.  A  variety  of  conditions  may  occur  which 
make  the  previous  selections  inappropriate  for  the  new  training  systems. 

For  example,  trade-offs  in  prior  analyses  may  have  been  based  on  fixed 
funds  which  required  selecting  less  than  the  most  effective  media,  whereas 
the  new  training  system  does  not  nave  cost  restrictions. 

If  specific  or  generic  task  statements  and/or  objectives  do  not  exist, 
the  user  must  go  to  one  of  the  other  two  forms  of  information  shown  in 
Figure  6  to  describe  the  traimnci  requirements.  The  new  job  for  which 
training  is  required  may  have  equipment  operation/maintenance  and  personnel 
descriptions.  Lacking  such  equipment  and  personnel  job  descriptions,  it 
may  also  be  possible  that  a  less  detailed  but  sufficient  description  ot  the 
overall  mission  with  references  to  the  job  to  be  trained  can  be  utilized. 

In  either  case,  the  user  needs  to  define  a  preliminary  course  outline  which 


^See  footnote  2  on  page  7. 
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contains  the  topics  and  sequence  of  topics  with  time  to  train  by  topic  if 
possible.  Then  the  user  can  determine  the  appropriate  attributes  required 
of  media  based  on  the  broad  topics  in  the  course  of  training.  This,  of 
course,  is  in  lieu  of  the  availability  of  objectives.  It  is  always  desirable 
to  select  media  for  each  objective  when  the  information  is  available  in  the 
form  of  objectives. 

The  following  section  describes  the  design  and  operation  of  the  prototype 
media  selection  model  in  terms  of  both  the  five  goals  and  the  five  functions 
established  for  the  automated  aid  to  ISD. 
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SECTION  III 


AUTOMATED  INSTRUCTIONAL  MEDIA  SELECTION  (AIMS) 

As  described  in  the  previous  section,  a  framework  was  established  to 
guide  the  development  of  AIMS.  The  framework  consisted  of:  (1)  the  goals 
defined  for  an  automated  aid  to  media  selection,  (2)  the  character! sties  of 
an  adequate  media  selection  model  identified  from  the  literature,  and  (3) 
an  analysis  of  the  functions  to  be  performed  in  automated  media  selection. 

This  section  discusses  two  aspects  of  AIMS.  First,  it  summarizes  the 
design  of  AIMS  in  terms  of  the  media  selection  functions  it  performs.  Second, 
it  illustrates  the  flexibility  and  power  of  AIMS  in  terms  of  the  five  goals 
established  for  automated  media  selection. 

DESIGN  AND  OPERATION  OF  AIMS 

The  automated  aspects  of  media  selection  depend  on  the  judgments  of  ISD 
personnel  who  must  determine  what  the  training  requirements  are  and  how  well 
media  generally  meet  such  requirements.  The  judgments,  based  upon  learning 
theory,  subject  matter  experience,  media  guidelines  and  other  "principles," 
cannot  be  automated  well.  They  must  be  left  to  human  judgment  as  in  the  case 
with  the  media  selection  models  reviewed  earlier. 

However,  the  basic  logic  in  all  of  the  media  selection  models  reviewed 
is  that  of  matching  the  required  characteristics  of  instructional  events  to 
the  attributes  of  media  available.  If  a  particular  medium  has  all  of  the 
attributes  needed  to  satisfy  the  intended  instruction,  that  medium  becomes 
a  candidate.  Those  media  not  meeting  the  requirements  are  discarded  from 
the  list.  The  AIMS  system  is  designed  to  automate  this  aspect  of  media 
selection.  Other  factors  such  as  organizational  constraints,  policy,  and 
costs  are  considered,  but  the  logical  matching  of  the  required  characteristics 
to  media  attributes  is  central. 

The  AIMS  system  is  designed  to  provide  rapid  access  to  a  data  base  of 
media  and  media  combinations,  cross-referenced  with  instructional  character¬ 
istics.  The  system  allows  a  user  to  input  the  instructional  characteristics 
required  for  a  given  training  objective,  and  the  computer  then  selects  media 
which  can  deliver  the  required  characteri sties,  scores  the  training  effective¬ 
ness  of  each  medium,  rank-orders  the  selected  media/media  combinations 
according  to  their  effectiveness  and  prints  out  the  results.  The  computer 
performs  these  tasks  many  hundreds  of  times  faster  than  humanly  possible. 

Instructional  characteristics  such  as  equipment  availability,  equipment 
costs,  task  difficulty,  task  frequency,  and  stage  of  learning  may  be  included 
as  part  of  the  selection  process.  Since  the  instructional  attributes  do  not 
have  to  relate  to  effectiveness  variables  per  se,  it  is  possible  to  perform 
trade-offs  for  a  variety  of  factors  and  constraints.  The  system  is  totally 
flexible  so  that  the  media  and  media  combinations  as  well  as  the  selection 
criteria  themselves  are  determined  by  the  requirements  and  constraints  of 
the  project  at  hand  and  not  general  notions  of  media  effectiveness.  Procedures 
for  using  the  AIMS  system  are  detailed  in  the  User's  Guide  in  the  Appendix  of 
this  report.  A  summary  of  the  major  steps  is  presented  here. 
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Five  functions  of  media  selection  were  previously  discussed.  They  are: 

(1)  problem  analysis,  (2)  definition,  (3)  information  analysis,  (4)  matching 
and  (5)  trade-offs.  Of  these,  the  AIMS  system  is  of  greatest  assistance  in 
the  matching  and  trade-offs  functions.  The  system  also  assists  in  the 
function  of  information  analysis  by  reducing  the  time  to  record  and  store 
the  results  of  the  analysis  of  training  objectives.  As  described  earlier, 
depending  on  the  stage  of  front-end  analysis,  information  analysis  can  also 
take  the  form  of  generic  objectives,  functional  hardware  analysis,  personnel 
job  descriptions,  or  mission  analysis.  The  AIMS  system  is  designed  to 
analyze  all  of  these  information  sources  in  an  efficient  manner.  The  function 
of  problem  analysis  and  definition  must  be  performed  by  ISD  personnel  in 
developing  an  AIMS  data  base  for  a  media  selection  problem.  The  steps  in 
using  AIMS  are: 

a.  _ Determine  the  Media  to  be  Considered.  The  user  cf  the  system  should 

first  make  a  list  of  all  the  media,  or  combinations  of  media,  to  be  included 

in  the  pool.  It  is  advisable,  at  this  stage,  to  allow  a  wide  range  of  alterna¬ 
tives  for  a  given  project.  If  need  be,  the  pool  can  easily  be  narrowed  if  a 
shorter  version  is  desired.  AIMS  allows  the  definition  of  any  pool  of  up  to 
90  media  and  90  attributes.  After  input  of  the  media  pool  AIMS  will  print 
the  media  pool  list.  Figure  7  shows  a  sample  media  pool  listing.  It  is  a 
broad  list  including  both  cognitive  and  hands-on  type  media.  Once  you  have 
determined  the  media  relevant  to  your  own  training  requirements,  you  can  begin 
the  next  phase. 

b.  Determine  the  Instructional  Characteristics.  Defining  the  instruc- 
tional  characteristics,'  or  "attributes"  as  they  are  also  called  is  an  important 
task.  Figure  8  presents  a  sample  list  of  attributes.  Note  how  the  attributes 
are  clustered  into  groups  of  similar  characteristics:  (1)  Display  character¬ 
istics,  (2)  Response  mode,  (3)  Evaluation  mode,  (4)  Feedback,  and  (5)  Special 
Requirements.  Breaking  the  attributes  into  small  categories  like  this  simpli¬ 
fies  the  task  of  creating  the  media  pool  and  helps  make  the  table  itself  a 

bit  more  legible.  At  this  point,  observe  Figure  9,  which  shnws  a  sample 
completed  media  pool.  Note  how  the  media  are  listed  down  the  "left-hand  column, 
and  the  attributes  are  listed  across  the  top  row,  creating  a  media-by-attributes 
matrix.  Creating  such  a  table  allows  the  next  phase  to  proceed. 

c.  Cross-Reference  Media  and  Their  Attributes.  For  each  medium  listed, 
the  analysts  will  then  weight  its  ability  to  deTiver  each  instructional  char¬ 
acteristic.  The  weighting  is  made  with  a  single  number,  between  0  and  5.  A 
"0"  weighting  indicates  that  the  medium  has  no  capability  for  delivering  an 
instructional  attribute.  A  "5"  indicates  that  the  medium  is  very  capable  of 
handling  a  particular  characteristic.  Note  in  Figure  9  that  the  medium 
"audio-tape"  is  ranked  "5"  under  the  attribute  "audio-voice,"  indicating  the 
strong  relationship  between  the  two.  "Workbook,"  of  course,  is  ranked  "0" 
under  the  audio-voice  capability,  since  it  has  no  ability  for  delivering  this 
characteristic. 

The  user  of  the  system  has  the  option,  of  course,  to  use  any  media  or  any 
instructional  characteristic  relevant  to  the  needs  of  the  project  at  hand. 

Other  instructional  characteristics  which  might  be  used  in  the  data  base  include 
availability  of  equipment,  cost  of  equipment,  task  difficulty,  stage-of-learning , 
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ATTRIBUTES  LISTED  ON  THE  FILE  :  pool 


*  I.  DISPLAY  CHARACTERISTICS 

1 .  Aud  io  (voiced 

2 .  Aud  io  ( amb  i ant i 

3.  Tactile  cues 

4.  Kinesthetic  cues 

5.  Verbal/text 

8.  Pictorial 

7.  Drawing  s/d iag rams 

8 .  Motion 

9.  Color 

*  II.  RESPONSE  MODE 

10.  Verbal/written 

11.  Po int- touch-mar k 

12.  Manipulate 

13.  Multiple  choice 

*  III.  EVALUATION 

14.  Instructor 

15.  Automated 
15.  Pe  e  r 

17.  Intrinsic 

*  IV.  FEEDBACK 

19.  Immediate  response 

19.  Immediate  error 

20 .  Pe  r iod  ic 

21.  Post  session 

*  SPECIAL  REQUIREMENTS 
72.  Crew/ team  interaction 

23.  Environmental  condition 

24.  Physical  motion 

25 .  Time  var i ab i 1 i ty 


Figure  8.  A  sample  listing  of  instructional  characteristics. 
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Figure  9.  A  sample  compie:  i  media  pool  with  each  medium's 
ability  to  del  nor  each  characteri stic  rated 
on  a  0-5  scale. 
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or  task  frequency,  to  name  a  few.  For  instance,  CAI ,  would  probably  be  ranked 
"0"  under  the  attribute  "low  cost,"  or  "5"  under  the  characteristic  "availability" 
should  one  be  available  at  the  proposed  training  site.  Similarly,  actual 
equipment  and  simulators  might  rate  high  on  the  dimensions  of  "task  difficulty" 
or  “task  frequency,"  so  that  the  more  difficult  and  frequent  tasks  are  assured 
simulator  training.  The  instructional  attributes  do  not  i  ave  to  relate  to 
effectiveness  variable  per  se  and  this  is  why  trade-offs  involving  cost, 
availability,  reliability  and  other  factors  are  possible. 

Once  the  media  pool  has  been  created,  it  needs  to  be  entered  into  the 
computer  as  a  data  base.  The  AIMS  system  provides  programs  to  allow  easy 
entering  of  these  data.  The  User  Guide  details  the  procedures.  Once  that  task 
is  complete,  the  user  needs  to  create  selection  worksheets  for  matching 
instructional  characteristics  to  objectives.  The  system  provides  for  automatic 
generation  of  such  a  worksheet.  The  worksheet,  of  course,  is  directly  tailored 
to  the  media  pool  from  which  it  was  created.  Figures  10  and  11  show  selection 
worksheets  from  two  different  media  selection  problems  which  were  produced  by 
the  computer.  AIMS  assists  in  the  Information  Analysis  function  of  media 
selection  by  reducing  the  time  to  record  and  store  the  results  of  learning 
objectives  analysis.  It  should  be  noted  also  that,  while  we  speak  of  learning 
objectives  analysis  here,  the  analysis  could  be  based  on  sets  of  objectives, 
generic  objectives,  hardware  functional  analysis,  mission  analysis  or  any  other 
information.  This  allows  one  to  use  the  system  regardless  of  the  state  of 
front-end  analysis,  as  will  be  detailed  later. 

As  in  moaels  discussed  earlier  in  the  report,  the  instructional  designers/ 
developers  need  to  use  the  worksheets  to  indicate  the  instructional  characteristics 
required  for  effective  and  efficient  delivery  of  instruction.  Since  this  is  a 
judgmental,  "thinking"  task,  the  worksheets  are  provided  to  allow  designers 
to  work  "off-line,"  not  consuming  unnecessary  computer  time  at  this  stage. 

Once  all  of  the  instructional  characteristics  for  a  given  group  of  objectives 
’  •-  been  determined,  these  data  need  to  be  entered  into  the  computer.  This  is 

an  easy  task;  complete  instructions  are  provided  in  the  User's  Guide.  Basically, 
the  person  inputting  data,  (a  fast  10-key  operator  might  be  best),  types  in  the 
objective  number  and  the  numbers  of  the  instructional  attributes  checked  off  on 
the  worksheet  for  that  objective.  The  computer  then  goes  to  work  in  several  ways: 

a.  Determines  media  which  have  the  required  characteristics. 

b.  Determines  how  well  eacn  'edium  handles  required  characteristics. 

c.  Scores  and  rank  orders  mecTa  ,s  to  their  appropriateness  for  the 
given  objective. 

d.  Records  how  many  times  each  medium  is  selected. 

e.  Stores  objectives  with  selected  media  on  data  base  of  objectives. 

The  computer  performs  these  tasks  many  hundreds  of  times  faster  than  would 
be  humanly  possible. 
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SELECTION  WORKSHEET  FOR  USE  WITH  THE  FILE  :  pool 

OBJECTIVE  NUMBER  :  _ _ _ 

OBJECTIVE: 


Put  a  check  in  the  boxes  next  to  required  attributes. 

*  I.  DISPLAY  CHARACTERISTICS  *  III.  EVALUATION 


1  1 

1.  Audio  (voicel 

1  1  14.  Instructor 

1  1 

2.  Audio  (ambienti 

1  1  15.  Automated 

1  1 

3.  Tactile  cues 

i  I  1A.  Peer 

1  1 

4.  Kinesthetic  cues 

1  i  17.  Intrinsic 

5.  Verbal/text 

*  IV.  FEEDBACK 

1— 1 

A.  Pictorial 

!  |  18.  Immediate  response 

7.  Drawings/diagrams 

i  1  19.  Immediate  error 

8.  Motion 

1  !  20.  Periodic 

l~l 

9.  Color 

1  1  21.  Post  session 

II. 

RESPONSE  MODE 

*  SPECIAL  REQUIREMENTS 

1  1 

10.  Verbal /writ ten 

!  1  22.  Crew/team  interact 

1  1 

11.  Fb int- touch-mar k 

j  1  23.  Environmental  cond 

I  1 

12.  Manipulate 

1  1  74.  Physical  motion 

1  1 

13.  Multiple  choice 

i  1  25.  lime  variability 

Figure  10.  A  saimle  selection  worksheet  from 
a  media  pool  entitled  "Pool." 
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SELECTION  WORKSHEET  FOR  'JSE  WITH  TOE  FILE  :  b:dass 

OBJECTIVE  NUM3ER  :  _ _ 

OBJECTIVE: 


Put  a  check  in  the  boxes  next  to  required  attributes. 


*1.  VISUAL  DISPLAYS 

! _ i  1.  single/clear  classic 

1 _ !  2.  single/clear  incomplete 

i _ I  2.  multiple/clear  classic 

I _ I  4.  multiple/ clear  incomplete 

I _ !  5.  single/ clear  confusing  lines 

i _ I  6.  single/peculiarities 

I _ 1  7.  multiple/clear  confusing  lin 

I _ I  S.  multiple/peculiarities 

*11.  AURAL  DISPLAYS 

I _ I  9.  single/clear  classic 

I _ I  10.  single/ clear  incomplete 

I _ I  11.  multiple/ clear  classic 

I _ I  12.  multiple/clear  incomplete 

I _ I  13.  .single/clear  confusing 

I _ I  14.  single/peculiarities 

I _ I  15.  multiple/ cl ear  confusing 

I _ I  16.  multiple/peculiarities 

♦III.  RESPONSE 

! _ I  17.  communication:  Senso 

I _ I  18.  communication :TACCO 

I _ I  19.  communication:  pilot 

I _ I  20.  communication: 

I _ I  21.  tactical  procedures 

I _ !  22.  analysis:  non-console 

I _ !  23.  analysis:  console 

I _ I  24.  display  option 

I _ !  25.  multiple  aonobuoy  monitorin 


1 _ I  26.  non-operating  response 

*IV.  MEASUREMENT 

I _ |  27.  overtimes 

I  I  28.  console  error  events 

I  j  29.  non-console  error  events 

I  I  30.  non-operating  error  events 

I  I  31.  GALT  sequence 

I _ I  32.  debrief  replay 

I  I  33.  time  and  success  summaries 

I  I  34.  display  option 

I _ |  35.  multiple  sonobuoy  monitorin 

*V.  STRATEGIES 

|  |  36.  interaction : Senso 

I  1  37.  interaction:  TACCO 
I  I  38.  interaction:  pilot 
I  I  39.  interaction:  other 

t _ I  40.  overtime  feedback 

1  1  41.  console  error  feedback 

I _ |  42.  non- con so 1 e  error  feedback 

I _ I  43.  non-operating  error  feedbac 

I _ 1  44.  change  LQFARgram  difficulty 

i _ I  45.  replay 

I _ I  46.  skip  event 

I _ I  47.  freeze 

I  1  48.  time  compression 
*VI.  ADDITIONAL  ATTRIBUTES 

1 _ |  49.  visual  data  display 

I  I  50.  aural  data  display 


Figure  11.  A  sample  worksheet  from  a 

media  pool  entitled  "DASS. " 
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Figure  12  shows  a  sample  media  selection  made  by  the  AIMS  model.  Note 
that  the  media  are  ranked  according  to  "soecific"  rating.  This  is  a  rating 
of  how  well  this  medium  handles  tne  characteristics  specific  to  the  objective. 
The  "general"  rating  is  the  average  of  the  quality  ratings  assigned  to  all  of 
the  instructional  character! sti cs  for  that  medium.  A  comparison  of  these 
ratings  indicates  the  degree  to  which  the  capabilities  of  that  medium  can  be 
exploited  for  a  given  objective.  For  example,  the  first  choice  in  Figure  12, 
"Op. Equip/Op.  Environment,"  sno*s  that  tne  medium's  attributes  matched  up  witr 
attributes  required  by  the  objective  for  a  highest  "specific"  rating  of  "E.1 
The  general  rating  (average  of  ell  ratings),  however,  for  this  medium,  was  4.4 
In  this  case,  the  medium's  attributes  matched  the  requirements  of  this  objec¬ 
tive  very  well.  The  opposite  was  true  of  medium  choice  #6,  "2D  Mock-up."  Ir. 
this  case,  the  average  general  rating  for  the  medium  was  3.36,  but  for  this 
specific  objective  the  rating  war  1.33,  showing  that  the  attributes  of  the 
medium  did  not  match  well  to  those  required  by  the  objective. 

GOALS  OF  AIMS 

Like  all  other  human  activities  requiring  analytic  thinking,  media  selec¬ 
tion  can  be  procedura  1  ized  and  aut:m?.ted  only  to  a  limited  extent.  ISO  per¬ 
sonnel  must  always  remember  chat  the  system  is  only  as  useful  as  tnei r  ar-lyt'. 
abilities  are  able  to  exploit  it.  The  following  discussion  of  the  five  cc-Ii 
of  the  AIMS  system  is  provided  to  further  explain  the  power  of  AIMS  to  assist 
ISD  personnel.  It  is  beyonc  the  current  scope  to  include  step-by-step  pro¬ 
cedures  for  the  examples  below.  Tney  are  intended  only  as  hints  and 
clarification  to  the  AIMS  user. 

MAXIMIZE  THE  USE  OF  PERTINENT  INFORMATION.  Ar.v  given  problem  may  be  approa  . •>.■ 
by  gathering  and  using  all  relevant  information  or  by  simplifying  the  effcrt 
using  a  small  subset  of  •information.  Media  models  in  the  literature  reflect 
from  one  end  of  the  continuum  "information  use"  to  the  other.  Either  aop^c ac¬ 
ts  possible  witn  AIMS  because  tne  user  defines  how  much  information  is  re'evon 
as  part  of  the  problem  analysis.  The  user  determines  which  and  how  many  cttri 
butes  should  be  derived  from  the  analysis  of  objectives.  The  user  determines 
what  should  be  included  in  the  media  pool.  The  user  decides  whether  an  existi 
media  selection  model  such  as  those  reviewed  earlier  satisfies  the  problem 
criteria. 

A  major  advantage  of  the  AIMS  system  is  that  large  amounts  of  information 
(attributes,  media  pool  items)  can  be  efficiently  input  into  a  computer- 
managed  data  base  and  utilized  at  a  tremendous  time  savings  over  a  manual 
effort.  In  fact,  when  a  large  amount  of  information  is  used  manually  it  can 
be  unwieldy,  especially  difficult  for  those  without  extensive  media  expertise, 
and  time  consuming.  The  AIMS  system  allows  the  user  to  define  the  specific 
media  selection  model  required  during  the  problem  analysis  and  definition  step 
Information  analysis  is  then  aided  by  allowing  the  user  to  input  the  coded 
analysis  of  objectives  into  the  data  base  instead  of  having  to  sift  through 
them  one  by  one  to  perform  matching  and  trade-offs.  Likewise,  because  all 
information  is  now  on  the  data  base  the  right  command  matches  the  analysis 
results  for  each  objective  with  each  medium,  summarizes  each  medium's  rank 
across  objectives,  and  prints  out  the  results  in  tabular  form.  Trade-offs 
can  easily  be  made  by  adding  or  changing  attributes  or  media  to  reflect  a 
modification  of  the  problem  statement  (e.g.,  what  if  we  eliminate  this 
instruction  that  requires  significant  amounts  of  audio?  or,  if  we  change  the 
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03JBCTIVE  #•>.?. 3. 4. 2.0 


MEDIUM 

SPECIFIC 

RATING 

GlNERAL 
RAT  DIG 

1 

*.  . 

Op.  Equip.  /Op.  Ehv ironma'it 

5.00 

4.43 

2 . 

Op.  Equip. /Lab 

4.33 

4.35 

3. 

Procedural  Trainer 

4.33 

4.  Sa 

4. 

Part  task  trainer 

3 . 67 

4.13 

5. 

3D  'tock-up 

3.67 

4. 36 

6. 

2D  r-tock-up 

1.33 

5 . 3o 

Figure  12.  A  sample  madia  selection  made  by  the  AIMS  model 
for  a  single  objective 
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instructional  setting  to  a  deployed  site  instead  of  a  schoolhouse  classroom, 
should  we  eliminate  media  which  cannot  be  supported  easily?) 

Most  of  the  labor  in  matching  and  trade-offs  is  computerized.  It  is 
performed  at  a  fraction  of  the  time  required  if  ISO  personnel  were  to  do  the 
same  thing  manually.  Because  of  the  efficiency  added  by  AIMS,  the  user  can 
design  the  media  selection  model  to  meet  the  problem  with  much  less  concern 
as  to  how  much  information  to  include  and,  thereby,  maxi  mi ze  the  information 
pertinent  to  media  selection. 

PROVIDE  FLEXIBILITY  TO  MODIFY  THE  MODEL.  One  problem  with  many  media  models 
is  that  all  media  selection  problems  are  treated  equally.  The  models  imply 
that  the  same  attributes  and  media  pool  can  be  used  for  almost  all  problems. 
This  is  not  true.  For  example,  one  situation  may  be  concerned  only  with 
selecting  from  currently  available  media  which  can  be  supported  by  existina 
in-house  resources;  while  another  case,  starting  without  any  support  resources 
or  restrictions,  may  be  open  to  all  possible  media. 

AIMS  provides  flexibility  by  allowing  the  user  to  change  the  media 
pool  and/or  the  media  attributes.  For  example,  the  user  may  wish  to  aac  a 
particular  medium  to  the  list  (emerging  videodisc  system)  or  add  a  capability 
assigned  to  a  particular  medium  (touch  panel  to  a  computer  assisted  instruc¬ 
tion  system).  These  user  functions  suggest  the  need  to  update,  change,  and 
add  to  the  media  selection  data  base.  The  recurring  problem  with  current 
media  selection  models  is  the  broadness  of  the  media  selections  and  the 
rigidity  of  the  media  pools.  The  possibility  of  creating  a  universal  media 
pool  seems  nil.  Media  pools  are  going  to  vary  in  terms  of  the  media  to  be 
considered,  the  relevant  instructional  characteristics,  the  level  of 
specificity,  and  changing  technologies  and  media  formats. 

Without  any  special  programming  expertise,  the  AIMS  user  may  add  ex¬ 
change  media  and  attributes,  or  create  entirely  new  media  pools,  without 
compromising  the  selection  process  itself.  The  emphasis  on  flexibility  in  to_ 
AIMS  system  makes  it  widely  applicable  to  a  variety  of  training  system 
development  projects.  It  satisfies  the  requirements  for  growth  potential 
and  allows  for  fulfillment  of  the  next  three  goals  to  be  discussed. 

CONSIDER  STUDENT  STAGE  OF  LEARNING.  There  is  an  overlap  between  this  goal 
and  the  goal  of  allowing  better  selection  capability  for  hands-on  training 
devices.  Generally,  as  the  stage  of  learning  becomes  more  advanced  there  are 
more  requirements  for  hands-on  training.  By  identifying  the  stage, of 
learning  first,  selection  may  be  directed  toward  the  applicable  hands-on 
training  requirements.  Procedures  for  using  AIMS  to  consider  student  stage 
of  learning  are  illustrated  below  with  three  examples.  Section  IV  of  this 
report  also  provides  an  example  of  a  situation  considering  stage  of  learning 
which  was  implemented  on  AIMS.  Specific  attributes  of  hands-on  training 
devices  as  they  relate  to  media  selection  using  AIMS  are  discussed  under 
Goal  4. 

To  consider  stage  of  learning  in  the  media  selection  process,  the 
concept  is  incorporated  as  an  instructional  characteri stic  which  then  enables 
the  computer  to  select  media  on  that  characteri stic.  The  User's  Guido  in 
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the  Appendix  provides  the  necessary  step-by-step  information  for  adding 
instructional  characteristics  to  the  media  pool.  However,  the  definition 
of  student  stages  of  learning  is  dependent  upon  the  training  system  and 
instructional  developers. 

In  one  example,  the  stages  of  learning  at  a  fleet  readiness  squadron 
could  be  defined  as:  (1)  familiarization,  (2)  memory,  and  (3)  actual  use, 
with  additional  substages  defined  by  the  content  of  information  to  be 
remembered  and  used.  Substages  are  thus  often  defined  as  facts,  concepts, 
rules  and  procedures.  This  scheme,  or  a  variation  of  it,  has  been  used  in 
several  Navy  aircrew  curricula  developments  to  classify  learning  objectives. 
After  these  attributes  are  included  in  the  data  base,  each  medium  in  the 
pool  would  be  rated  by  how  well  it  could  provide  for  each  stage  of  learning. 
For  example,  a  simplified  diagram  of  an  operator's  console  might  be  usable 
for  familiarization,  a  workbook  and  technical  manual  for  acquisition  of  in¬ 
formation,  and  a  simulator  trainer  for  using  the  information  under  various 
conditions. 

The  LAMPS  MARK  III  model  described  in  Section  II  provides  another 
example  of  determining  student  stage  of  learning.  In  this  model,  the  first 
step  is  to  classify  the  objective  as  either  "hands-on"  or  cognitive.  In  the 
second  step,  the  level  of  learning  for  each  hands-on  objective  is  determined. 
As  shown  in  Table  2  in  the  left  column,  there  are  five  levels  of  learning 
for  "hands-on"  objectives. 

The  levels  of  learning  are  then  used  to  prioritize  the  media  selected. 
Table  3  represents  a  priority  assignment  table  used  for  training  devices. 

The  letters  across  the  top  pertain  to:  (A)  operational  equipment-operational 
environment,  (B)  operational  equipment-laboratory  equipment,  (C)  team  trainer, 
(D)  procedural  trainer,  (E)  part-task  trainer,  (F)  three-dimensional  mock-up, 
and  (G)  two-dimensional  mock-up.  The  numbers  indicate  the  rank  (from  1-7 
with  7  the  lowest)  of  a  device's  capability  to  provide  the  training  for  a 
level  of  learning  based  on  the  "complexity/fidelity"  required  and  procurement 
costs. 


The  TAEG  model  described  in  Section  III  provides  yet  another  example  of 
stage  of  learning  analysis  which  could  be  implemented  on  AIMS.  It  illustrates 
different  techniques  for  taking  into  account  stages  of  learning  than  those 
used  in  the  LAMPS  MARK  III  model.  The  TAEG  model  makes  the  distinction  be¬ 
tween  training  stages  and  training  tasks.  Each  stage  consists  of  fundamentally 
similar  tasks.  Stages  are  ordered  so  that  prerequisite  skills  are  acquired 
before  advanced  skills  are  introduced.  The  training  tasks  are  therefore 
organized  into  a  preliminary  curriculum  outline  which  should  consist  of  a 
series  of  stages.  Table  4,  taken  from  the  TAEG  report,  illustrates  the  dual 
categorization  by  both  stage  and  tasks  within  a  stage.  Note  that  tasks 
within  a  stage  (center  column)  also  consist  of  two  or  more  primary  task 
categories  which  indicate  the  type  of  learning  required.  The  procedures 
for  selecting  the  appropriate  media  include: 


T7 


See  footnote  7  on  page  11. 
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TABLE  2.  "HANDS-ON"  LEARNING  LEVELS  USED  IN  LAMPS  MARK  III 
MEDIA  SELECTION  MODEL 


Classification 

Familiarity 


Typical  Verbs 

Identify,  Select, 
Locate,  Touch,  List 


General  Description 

The  trainee  learns  general 
context  knowledge  of  the  job 
and  mission  to  be  implemented 
and  will  learn  to  identify, 
locate,  perceive  the  inter¬ 
related  personnel,  equipment, 
and  material  with  which  he 
will  interact. 


Perform  Procedure 


Perform  Job 


Configure,  Remove, 
Assemble,  Assign, 
Retrieve,  Prepare, 
Inspect,  Open, 
Close 


Classify,  Track, 
Launch,  Fly, 
Land,  Repair, 
Fix,  Transmit,  • 
Decide,  Align, 
Operate,  Control 


The  trainee  learns  the 
sequences,  movements,  and 
mental  processes  to  be  fol¬ 
lowed  in  order  to  perform 
elements  of  his  job.  These 
procedures  are  generally 
learned  to  an  unaided  but 
elementary  skill  level. 

The  trainee  learns  to  per¬ 
form  all  procedures,  pro¬ 
cesses,  and  decisions 
required  in  a  job  at  a 
skill  level  required  oper¬ 
ationally.  This  type  of 
of  learning  involves  varied 
job  experiences  and  related 
concurrent  activities. 


Perform  Mission 


Perform  Under 
Unusual /Degraded 
Conditions 


Comply,  Order, 
Acknowledge , 
Confirm,  Request, 
Report,  Relieve, 
Share,  and 
Combined  Action 
Verbs 


Verbs  are  similar 
to  Perform,  Job, 
Mission.  Condi¬ 
tions  must  be 
evaluated. 


The  trainee  learns  to  per¬ 
form  the  job  in  coordination 
and  communication  with, 
dynamic  interaction  with, 
other  functional  personnel 
(real  or  simulated).  This 
typo  of  learning  requires 
highly  proficient  job  skills 
for  effective  training. 

The  trainee  learns  to  cope 
with  the  unexpected  and 
unusual  events  that  may 
occur  in  operation  such 
as  equipment  malfunction, 
incorrect  information, 
short  handed,  extraordinary 
environments.  This  type  of 
learning  also  has  as  a  pre¬ 
requisite  highly  proficient 
job  skills. 
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TABLE  3.  PRIORITY  ASSIGNMENT  "HANDS-ON"  MEDIA  POOL  IN 
LAMPS  MARK  III  MEDIA  SELECTION  MODEL. 


A 


Familiarity  7 

Procedural  Performance  7 

Job  Performance  4 

Mission  Performance  2 

Unusual  Event  2 


B  C  D  E  F  G 

3  6  5  4  2  1 

5  6  4  3  2  1 

5  3  12  6  7 

5  1  3  4  6  7 

7  1  3  4  5  6 
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TABLE  4.  CATEGORIZATION  OF  TASKS  IN  THE  FAMILIARIZATION  STAGE  IN  THE 
TA-4J  ADVANCED  JET  NAVY  UNDERGRADUATE  PILOT  TRAINING  PROGRAM. 


Stage 

Tasks  Within  Stage 

) 

Primary  Task  Categories 

Familiarization 

Cockpit  Procedures 

Recalling  Procedures 

Non-Verbal  Identification 
Positioning  &  Serial  Movement 

Preflight  Inspection 

Recalling  Procedures 

Non-Verbal  Identification 
Making  Decisions 

Normal  Operating 
Procedures 

! 

Recalling  Procedures 

Non-Verbal  Identification 
Continuous  Movement 

Positioning  and  Serial  Movemen 

Nosewheel  Steering  and 
Taxi  Procedures 

Recalling  Procedures 

Non-Verbal  Identification 
Continuous  Movement 

Oral  Verbalization 

I 

j 

Take-Off /Climb/Level 

Off 

Recalling  Procedures 

Non-Verbal  Identification 
Continuous  Movement 

Positioning  and  Serial  Movemen 
Oral  Verbalization 

'  ~ .  1 

|  Aerobatics 

Recalling  Procedures 

Non-Verbal  Identification 
Continuous  Movement 


Landing  !  Recalling  Procedures 

Non-Verbal  Identification 
Continuous  Movement 
Positioning  and  Serial  Movement 
Oral  Verbalization 
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a.  Identifying  the  stage  and  task  type 

b.  Determining  the  guidelines  and  media  implications  for  the  different 
learning  task  categories,  and 

c.  Matching  the  implications  to  options  in  the  media  pool.. 

Details  of  each  step  may  be  found  in  the  TAEG  staff  study  report.*® 

ALLOW  MORE  DETAILED  SELECTION  OF  TRAINING  DEVICES.  Since  each  user  can 
create  his  own  media  pool,  extensive  listings  of  simulators,  actual  equipment, 
and  other  hands-on  type  media  and  their  attributes  are  fully  permissible. 
Fidelity  requirements  can  be  included  as  instructional  characteristics, 
thereby  directly  bearing  on  the  media  selected.  The  bias  toward  cognitive 
type  media  which  is  apparent  in  existing  models  can  be  eliminated. 

Media  selection  models  do  not  usually  provide  procedures  for  determining 
specific  simulator  requirements  and  matching  objectives  to  simulators  to  be 
used  for  training.  If  selection  criteria  are  identified,  they  are  not  usually 
comprehensive.  The  AIMS  approach  provides  a  tool  for  systematic  selection  of 
simulators  as  well  as  other  media.  The  use  of  stage  of  learning  attributes 
and  the  definition  of  specific  attributes  required  for  displays,  responses 
and  other  learning  task  elements  are  important  in  selecting  simulators. 

The  key  to  determining  when  and  how  simulators  should  be  designed  and 
used  in  training  is  in  the  concept  of  level  of  fidelity.  Gagne's  definition 
is: 


"A  simulator  is  generally  understood  to  be  a  kind  of  training  device 
which  has  a  high  degree  of  resemblance  to  operational  equipment, 
particularly  with  respect  to  the  display,  the  controls,  and  the  way 
one  affects  the  other  when  in  operation. "1$ 

According  to  Gagne,  training  simulators  have  three  characteristics  in  common, 
each  of  which  is  related  to  fidelity  and  must  be  taken  into  account  in 
media  selection.  The  characteristics  are: 

a.  A  representation  of  the  real  situation  in  which  operations  are 
carried  out 

b.  A  way  to  provide  desired  controls  over  the  represented  situation 


18 


See  footnote  4  on  page  10. 


19R.  Gagne, 
AFPTRC-TR-54-16. 
Research  Center, 


Training  Devices  and  Simulators:  Some  Resea rch  Issues 
(LackT and  AFB,  TX:  Air  Force  Personnel  and  Training 
Air  Research  and  Development  Command,  May  1954). 
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c.  The  omission,  by  design,  of  certain  parts  of  the  real  situation." 

The  key  question  in  decisions  of  when  and  how  to  use  a  simulator  in 
training  is  how  much  representation  to  provide.  Costs,  availability  of  real 
equipment,  and  other  practical  factors  enter  into  final  media  mix  decisions, 
but  these  are  factors  to  be  considered  at  a  later  stage  rather  than  during 
the  analysis  of  trailing  requirements.  As  pointed  out  in  previous  work  for 
the  NAVTRAEQUIPCEN,  there  are  some  guidelines  for  developing  and  using 
simulators,  but  they  have  not  been  formally  procedural ized  within  the  ISD 
process. 

The  concept  of  the  AIMS  system  is  to  allow  the  user  to  incorporate  specific 
information  on  skills  training  and  simulation  which  requires  higher  fidelity 
than  that  available  with  general  media.  Ultimately,  the  user  must  decide  how 
much  information  to  incorporate  to  satisfy  the  training  situation.  Figure  13 
provides  a  methods/media  framework  suggested  as  a  model  for- the  AIMS  user. 

This  framework  uses  many  of  the  same  general  ideas  as  the  LAMPS  MARK  III  model 
described  earlier.  However,  tne  AIMS  system  should  provide  improvements  by 
allowing  for  more  detailed  selection  of  training  devices  in  each  unique 
training  program. 

The  following  paragraphs  provde  a  brief  explanation  of  the  five  steps 
shown  in  Figure  13.  Where  appropriate,  statements  concerning  training  simula¬ 
tor  literature  are  made  to  assist  tne  AIMS  user  in  the  problem  analysis  and 
definition  functions.  Section  IV  illustrates  how  specific  fidelity  require¬ 
ments  were  handled  for  a  demonstration  project  using  AIMS. 

22 

Step  1  -  Analysis  of  Objectives.  Til  den  pointed  out  that  the  degree  of 
fidelity  required  for  simulators  is  a  consequence  of  the  training  objectives 
and  what  is  known  about  how  to  train  the  objectives  (referred  to  as  learning 
theory).  In  this  case,  learning  theory  means  what  is  known  about  information 
presentation,  perception,  feedback,  motivation,  transfer  of  learning,  ana 
other  methods/media  issues.  However,  the  major  question  to  be  answered  is 
how  to  relate  fidelity  requirements  to  a  training  objective. 

The  first  step  in  the  LAMPS  MARK  III  model  is  the  analysis  of  objectives 
for  a  "hands-on"  requirement.  The  hands-on  decision  is  made  by  determining 
if  a  representation  or  a  real  display  or  response  feature  is  required.  The 
model  proposed  here  carries  this  notion  further  in  order  to  utilize  more  of 
the  relevant  information  which  can  be  derived  from  analyzing  objectives. 

The  user  should  analyze  the  three  components  of  each  objective  (behavior, 
standard,  conditions)  since  each  of  these  components  provides' relevant  fide! iu 
information.  Conditions  tell  us  something  about  display  requirements.  For 


20 

See  footnote  19  on  page  46. 

21 

W.  W.  Prophet,  U.S.  Navy  Fleet  Aviation  Training  Development, 
NAVTRAEQUIPCEN  Technical  ReportTT7TC-0009-l .  (Pensacola,  FL:  Seville 
Research  Corporation,  March  1978). 

77 

C.  H.  Tilden,  Problems  in  Training  Equipment  Design,  (Nortn  American 
Aviation,  Inc.,  Space  and  Information  Systems  Division,  1962). 
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Ficurc  13.  Summary  of  proposed  methods/medi a  selection. 
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example,  if  a  LOFARGRAM  with  visual  cues  for  primary  harmonic  family  is 
indicated  in  an  objective,  then  the  display  fidelity  must  be  high  regardless 
of  the  presentation  medium.  Standards  provide  information  related  to 
response  and  recording  (performance  measurement)  requirements.  The  behavior 
tells  us  something  about  interactiveness  and  general  response  mode. 

Step  2  -  Fidelity  Level.  By  identifying  the  fidelity  requirements  for  the 
objectives,  the  user  can  then  determine  whether  training  devices  or  simulators 
are  required.  In  some  cases,  it  will  be  possible  for  both  training  devices 
and  classroom  media  to  perform  for  a  given  objective.  Step  2  requires  only 
that  a  decision  is  made  that  higher  fidelity  is  required  than  available  with 
conventional  media.  The  detailed  requirements  will  be  used  later  in  Steps 
3  and  4, 

Step  3  -  Classification.  In  a  typical  classroom  media  selection  model,  the 
Ftassification  of  objectives  by  learning  type  is  part  of  the  ISD  procedures. 
This  step  also  exists  in  the  AIMS  model.  It  is  suggested  that  classification 
by  stage  of  learning  also  be  done  wnere  the  learning  objectives  indicate  a 
higher  fidelity  device  may  be  required. 

A  variety  of  classif i cation  schemes  has  been  used  by  practitioners.  The 
categories  used  in  the  LAMPS  MARK  III  and  TAEG/TECEP  procedures  were  describee 
earlier  in  this  report.  The  categories  may  be  as  simple  as  "early,"  "middle,' 
and  "high"  stages.  The  categories  may  be  defined  by  agreement  that  a  soec i f i c 
task  belongs  to  a  category.  Definitions  of  categories  which  are  totally  clear 
may  be  difficult  to  achieve. 

Step  4  -  Determine  Strategies/Events.  Stages  of  learning  are  relevant  to 
selections  involving  trainers  and  real  equipment  because  they  aid  in  deter¬ 
mining  the  need  for  three  aspects  of  fidelity  discussed  by  Kinkade  and  Wheaton 
These  are:  (1)  equipment  fidelity,  (2)  environment  fidelity,  and  (3)  psycholo 
ical  fidelity.  These  three  aspects  of  fidelity  relate  to  the  design  of  tne 
trainer  and  control  of  the  representations.  By  incorporating  the  relevant 
fidelity  factors  for  a  training  situation  into  the  media  pool  and  attributes 
data  base  definition,  the  user  can  provide  for  more  detailed  selection  of 
training  devices  and  simulators. 

Research  done  on  how  fidelity  factors  relate  to  stages  of  learning 
indicate: 

a.  At  later  stages  of  learning,  specific  factors  are  required  peculiar 
to  the  stage.24 


R.  G.  Kinkade  and  G.  R.  Wheaton,  "Training  Device  Design,"  in  Humar 
Engineering  Guide  to  Equipment  Design,  ed.  H.  P.  VanCott  and  R.  G.  Kinkade 
^Washington,  DC:  American  Institutes  for  Research,  1972). 

24 

E.  A.  Fleishman,  "The  Description  and  Prediction  of  Perceptual -Mot o*- 
Skill  Training,"  in  Trainirc  Research  ard  Education,  ed.  R.  Glaser 
(Washington,  DC:  Office  of  Naval  Research,  196f). 
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b.  As  the  skill  level  proficiency  is  expected  to  increase,  the  fidelity 
of  the  learning  environment  should  increase.  25 

c.  In  early  stages  of  learning,  (such  as  learning  nomenclature)  a  high 
fidelity  learning  environment  is  inefficient^  and  may  actually  confuse  the 
trainee. 27  An  exception  to  this  might  be  the  learning  of  complicated  motor 
skills  where  the  training  device  is  designed  to  accommodate  varying  levels  of 
proficiency.  28 

d.  As  the  student  enters  a  "middle"  stage-of-learni ng  (more  task  integra¬ 
tion  and  higher  expected  proficiency)  fidelity  increases  for  such  factors  2% 

as : 

(1)  Noise  which  must  be  learned  as  irrelevant 

(2)  Guiding  cues  for  action 

(3)  Di scrimi nation  cues 

(4)  Time  relationships 

(5)  Task  and  event  loads 

e.  In  high  (final)  stages  of  learning,  fidelity  should  be  increased 
still  more  for  each  of  the  factors  under  "d"  above. ™  Practice  should 
simulate  with  accuracy  the  operational  environment  where  such  factors  are  the 
basis  for  task  performance. 

Step  5  -  Selection.  The  final  step  is  to  list  and  prioritize  the  trainer  and 
conventional  media  options  which  meet  the  training  objectives  requirements. 


G.  Kinkade  and  G.  R.  Wheaton,  "Training  Device  Design,"  in  Human 
Engineering  Guide  to  Equipment  Design,  ed.  H.  P.  VanCott  and  R.  G.  Kinkade 
(Washington,  DC:  American  Institutes  for  Research,  1972). 

26 

R.  B.  Miller*  Psychological  Considerations  in  the  Desiqn  of  Training 
Equipment ,  W ADC -TR - 54 - 553 TAD- 71  202) ,  "(Wright  Air  Devel oprolt  CeItTITw“rIqht- 
Patterson  AFB,  OH,  December,  1954). 

27 

G.  G.  Miller,  Some  Considerations  in  the  Design  and  Utilization  of 
Simulators  for  Technical  : raining,  AFHRL-TR-74-65  (Lowery  Air  Force  Base , 
Colorado,  Human  Resource  Laboratory,  August,  1974). 

Gagne,  Training  Devices  and  Simulators:  Some  Research  Issues, 

P.- PTRC-RT-54-16.  Lackland  AFB,  TX:  Air  Force  Personnel  and  Training  Research 
Center,  Air  Research  and  Development  Command,  May  1954. 

29 

E.  A.  Fleishman,  "The  Description  and  Prediction  of  Perceptual-Motor 
Skill  Training,"  in  Training  Research  and  Education,  ed.  R.  Glaser 
(Washington,  DC:  Office  of  Nava)  Research,  1961). 
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ALLOW  FOR  VARIATIONS  IN  Trtt  STAGl  OF  FRONT-END  AU» .LYSIS.  The  convent! ,  he I 
thinking  behind  media  selection  models  is  that  they  are  used  by  a  combi  nan 
of  instructional  designers  and  media  specialists  during  instructional  deve: 
ment.  Usually  included  in  me  "use"  scenario  is  the  assignment  that  a  joe/ 
task  analysis  has  been  perfc med  with  objectives  defined  and  arranged  in  a 
fashion  indicating  instructional  sequence. 

The  problem  arises  during  tne  conceptual  stage  of  new  weapon  system 
programs  when  this  in-'ormaticr  is  net  available  and  only  gradually  becomes 
available  as  the  frort-erc  analysis  for  training  is  performed.  Yet  procra: 
managers  must  make  decisions,  prior  to  a  full  front-end  analysis,  for  E-ait 
plans  which  include  billet,  budgetary,  and  support  resource  requi rements . 
least  tentative  decisions  rust  be  ’uae  early  in  a  program  on  media  (includi 
training  devices)  utilization.  In  many  cases,  this  has  been  an  almost  it; 
task  which  is  resolved  in  Qn  armchair  mode  with  best  guesses  and  little  bat 
ing  rationale  due  to  the  lack  of  inf creation. 

As  seen  in  Figure  mure  are  two  types  of  information:  (1)  tasi^  st 
ments  and  (2)  behavioral  objectives.  Each  of  these  two  information  tyx-s  " 
three  possible  levels  cm'  quality:  {' )  completely  analyzed  for  all  tasks  or 
objectives  (full),  (2;  analyzed  only  for  what  are  identified  as  new  or  urm 
system  functions  (system  specific),  anc  (3)  general  (generic)  task  stater e- 
which  could  apply  across  various  aircraft  as  derived  from  CASDAT  (or  some  .. 
source  if  it  exists  for  a  non-aircraft  system).  In  the  best  situation,  ..  * 
task  analysis  and  full  behavioral  analysis  would  be  available  on  which  to  li¬ 
the  media  selection.  As  one  or  both  of  the  information  types  becomes  lower 
in  quality,  from  completely  analyzed  to  system  specific  only  to  generic,  a 
grosser  determination  of  meric  requirements  is  made.  Figure  14  illustrates 
level  of  media  selection  specificity  that  might  be  obtained,  depending  „pc- 
the  quality  of  information  available. 31 

It  is  possible  to  mare  a  medic  selection,  regardless  of  tne.-  i -*c 
quality,  using  AIMS.  During  the  problem  analysis,  the  AIMS  user  s-od \ 
termine  what  level  of  information  is  available.  The  AIMS  system  cots 
the  difference,  but  the  user  should.  If  the  level  of  information  to  ix 
for  media/attribute  requirements  is  at  the  mission  phase,  equipment  Vf 
some  other  gross  leve;,  men  tne  definition  of  the  attributes  and  medic 
data  base  may  not  nave  to  be  extremely  detailed.  How  detailed  the  cate  ta. 
should  be  is  subject  to  the  preference  of  the  user,  based  on  the  problem 
Typically,  in  tne  early  stages  of  weapon  system  development,  only  estimate 
are  needed  and  men. a  selection  will  be  finalized  later  based  or  a  full 
end  analysis.  Fcweve-,  if  tne  early  selection  is  likely  to  determine  the 
media,  then  the  user  should  be  as  detailed  as  possible  in  defining  the 
attributes  and  media  pool  data  case. 


^ J.  Funaro,  "Front-End  Analysis  (FEA)  for  Emerging  Weapon  Systems,1 
paper  presented  at  the  DoS/DARPA  Conference,  Santa  Fe,  NM,  September  197a. 
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DEMONSTRATION  OF  THE  MODEL 

As  part  of  an  ongoing  project.  Instructional  Science  and  Development,  Inc. 
was  tasked  to  select  media  for  an  acoustic  sensor  operator  deployed  training 
system  (DTS).  The  project  represented  a  unique  media  selection  problem.  The 
training  required  high  fidelity  displays  so  a  trainer  capability  or  real  equip¬ 
ment  was  required.  However,  no  trainer  already  existed  which  could  satisfy  the 
requirements.  The  problem  illustrates  how  Goal  3,  consideration  of  stage  of 
learning,  and  Goal  4,  more  detailed  selection  of  hands-on  training  devices,  car, 
be  handled  with  AIMS.  The  purpose  of  this  section  is  to  provide  an  example 
of  how  the  AIMS  system  was  used  and  the  general  outcomes  of  this  application. 

The  description  is  organized  by  the  five  major  functions  of  media  selection 
defined  earlier. 

PROBLEM  ANALYSIS 

A  review  of  media  selection  models,  such  as  the  Interservice  Procedures 
Model  for  Instructional  System  Development,  did  not  result  in  finding  an 
existing  model  which  was  deemed  suitable.  An  approach  to  media  selection  wrier 
was  highly  specific  to  the  acoustic  sensor  operator  training  problem  statemert 
was  desired.  Most  media  selection  algorithms  are  more  concerned  with  academe 
or  learning  center  type  media.  In  addition,  the  instructional  characteristics 
which  are  used  to  describe  the  media  are  general  in  nature.  Typically,  they 
include  the  type  of  visual  (still  or  motion;  color  or  black  and  white;  pictures 
or  drawings),  audio  (voice  or  sound),  and  other  general  type  characteri sti cs . 
Furthermore,  most  media  selection  models  do  not  specifically  take  into  account 
stages  of  learning  and  advanced  proficiency  measurement.  Therefore,  it  was 
determined  that  a  procedure  which  provided  for  specific  fidelity  of  display 
and  response  decisions,  as  well  as  the  other  detailed  requirements  of  a  deployed 
training  system,  was  required. 

DEFINITION 

Table  5  lists  the  instructional  characteristics  which  were  determined  to 
be  relevant  to  DTS  media  selection.  The  characteristics  are  specific  to  the 
job  of  acoustic  sensor  operator  and  the  training  that  job  requires.  Note  that 
the  instructional  characteri sties  are  categorized  by  display,  response,  measure¬ 
ment,  and  instructional  strategy  requirements. 

The  display  requirements  were  taken,  in  part,  from  the  current  'procedures 
for  measuring  proficiency  of  acoustic  sensor  operators.  The  proficiency 
measures  cite  the  complexity  and  clarity  of  displays  during  analysis.  Eight 
proficiency  levels  for  analyzing  acoustic  data  from  visual  and  aural  data 
were  used  to  define  the  visual  and  aural  display  characteristics.  The  eight 
items  listed  under  each  of  the  categories  Visual  Displays  and  Aural  Displays 
are  abbreviated  names  for  the  proficiency  levels.  The  proficiency  levels  and 
associated  display  requirements  are  a  function  of  the  type  of  target,  its 
clarity,  and  number  of  signatures  being  analyzed.  Although  the  eight  proficient; 
levels  are  not  exactly  hierarchical  they  are  still  a  stage  of  learning  indicator 
and  reflect  a  categorization  recognized  by  operators. 
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TABLE  5.  INSTRUCTIONAL  CHARACTERISTICS  FOR  DTS  MEDIA  SELECTION 


VISUAL  DISPLAYS 

1.  single/clear  classic 

2.  single/clear  incomplete 

3.  multiple/clear  classic 

4.  multiple/clear  incomplete 

5.  single/clear  confusing  lines 

6.  single/peculiarities 

7.  multiple/clear  confusing  lines 

8.  multiple/peculiarities 
AURAL  DISPLAYS 

9.  single/clear  classic 

10.  si  r.gle/clear  incomplete 

11.  multiple/clear  classic 

12.  mul tiple/clear  incomplete 

13.  single/clear  confusing 

14.  single/peculiarities 

15.  multiple/clear  confusing 

16.  multiple/peculiarities 
RESPONSE 

17.  communication  Senso 

18.  communication:  Tacco 
1§.  communication:  p)  f  1  o  t 

20.  communication:  other 

21.  tactical  procedures 

22.  analysis:  nor, -console 

23.  analysis:  console 

24.  display  option 

25.  mult,  sonobuoy  monitoring 

26.  non-operating  response 


MEASUREMENT 

27.  overtimes 

28.  console  error  events 

29.  non-console  error  events 

30.  non-operating  error  events 

31.  GALT  sequence 

32.  debrief  replay 

33.  time  and  success  summaries 

34.  display  option 

35.  mult,  sonobuoy  moni tori ng 
STRATEGIES 

36.  interaction:  Senso 

37.  interaction:  Tacco 

38.  interaction:  pilot 

39.  interaction:  other 

40.  overtime  feedback 

41.  console  error  feedback 

42.  non-console  error  feedback 

43.  non-operating  error  feedback 

44.  change  LDI 

45.  replay 

46.  skip  event 

47.  freeze 

48.  time  compression 
ADDITIONAL  ATTRIBUTES 

49.  visual  data  display 

50.  aural  data  display 
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The  response  requirements  included  actions  taken  at  a  console,  as  well 
as  actions  which  would  normally  be  part  of  the  operator's  job  but  not  related 
to  the  console  procedures.  For  example,  instructional  characteristics  were 
included  which  relate  to  the  interaction  between  acoustic  sensor  operator  and 
other  crew  members,  since  concern  was  expressed  about  the  need  for  subteam  or 
team  type  interaction. 

These  types  of  responses  are  also  reflected  in  the  instructional  charac¬ 
teristics  categorized  under  measurement.  It  was  considered  important  to  be 
able  to  measure  the  sequence  of  actions,  the  timeliness  of  the  actions,  and  the 
success  of  the  actions  taiei .  The  instructional  characteri sties  in  the  measure¬ 
ment  category  relate  directly  to  the  display  and  response  characteristics. 

Finally,  the  instructional  characteristics  related  to  training  strategies 
are  listed.  Included  are  diagnostics  on  overtimes  (beyond  acceptable  standards 
in  objectives)  and  errors  rf  several  possible  types  as  well  as  feedback 
capability  for  these  errors.  The  capability  is  included  for  instructor  actions 
during  real  time  playing  of  the  scenario.  Freezing  the  exercise  when  a  trainee 
has  accomplished  the  objective  and  there  is  no  sense  in  continuing  to  run  the 
exercise,  and  replaying  some  portion  of  the  exercise  to  reinforce  or  provide 
for  practice  are  seen,  in  Table  5,  as  examples  of  instructor  actions  which  couia 
be  required. 

The  media  pool  for  training  of  acoustic  sensor  operators  at  an  advanced 
level  of  proficiency  need  to  reflect  the  entry  level  into  the  training  and 
the  constraints  of  the  training  environment.  The  entry  level  of  the  students 
is  fairly  high.  The  acoustic  sensor  operator,  at  a  deployed  setting,  has  been 
performing  his  job  for  some  period  of  time  in  the  actual  environment.  The 
training  media  must  have  a  fairly  high  level  of  fidelity  because  the  operator's 
job  is  dependent  upon  the  console  display  of  lines  representing  ocean  acoustics. 
It  was  determined  earlier  that  the  analysis  phase  of  the  acoustic  sensor's  job 
was  the  most  critical  for  training.  Therefore,  it  became  important  for  the 
incoming  information,  both  visual  and  aural,  to  be  of  sufficient  fidelity  during 
training  to  transfer  to  the  analysis  phase  of  the  job.  In  addition,  it  did  not 
seem  necessary  to  have  a  media  pool  with  such  commonplace  learning  center  media 
as  audio  tapes,  slide/sound,  filmstrip,  or  other  audiovisual  devices.  It  was 
more  suitable  that  the  deployed  training  system  have  support  media  more  in  tune 
with  storage  and  operation  limitations.  Some  of  the  deployed  settings  are  on 
board  ships  which  have  little  or  no  extra  room.  Therefore,  support  media  such 
as  printed  material,  both  for  students  and  instructors,  were  considered  more 
appropriate.  The  instructors  themselves  were  also  included  as  part'of  the 
support  media  since  they  play  an  important  part  in  the  actual  instructional 
events  as  well  as  the  evaluation  and  management  of  training. 

A  meeting  was  held  with  personnel  from  the  Navy  Personnel  Research  and 
Development  Center  (NAVPERSRANDCEN) ,  the  Naval  Air  Development  Center 
(NAVAIRDEVCEN) ,  and  the  contractor  to  define  possible  alternatives  for 
delivering  the  instruction.  Six  alternatives,  based  on  three  different  hard¬ 
ware/software  technologies,  were  defined  with  varying  capabilities  for 
satisfying  the  training  requirements.  The  technologies  included  the  use  of 
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onboard  tape  systems,  the  use  of  specially  built  station  mock-ups  or  table 
top  devices  with  tape  input,  and  the  use  of  acoustic  modules  for  generating 
signals  into  the  aircraft.  The  six  alternatives  are  surmarized  below: 

a.  Onboard  tape  system  wi thout  benefit  of  instructional  materials. 

b.  Onboard  tape  system  with  instructional  materials.  Use  of  multi¬ 
channel  capability  to  design  new  tapes  from  master  set. 

c.  Tape  system  (compatible  with  aircraft)  with  mock-up  of  operator 
station  and  instructor  console  for  real  time  exercising  control. 

d.  Tape  system  (compatible  with  aircraft)  with  table  top  trainer  having 
lower  procedural  fidelity. 

e.  Acoustic  module  system  with  full  dynamic,  real  time  control  of 
scenarios. 

f.  Acoustic  module  system  functionally  equivalent  to  "b"  above. 

Next,  ISO  personnel  and  subject  matter  experts  who  knew  the  capabilities 
of  the  onboard  tape  system  and  the  operator  task  requirements  determined  which 
characteristics  required  for  training  were  available  on  the  six  alternatives 
and  how  well  each  alternative  performed  on  that  characteristic.  Each  configura¬ 
tion  was  given  a  score  of  0  to  5  for  its  capability  to  provide  each  instruc¬ 
tional  characteristic.  The  matrix  in  Figure  15  summarizes  the  weights 
for  each  instructional  characteristic  for  each  media  alternative. 

INFORMATION  ANALYSIS 

A  selection  worksheet  was  generated  by  the  computer  for  matching  each 
objective  to  instructional  characteristics.  The  analysis  of  objectives  was 
performed  by  ISD  personnel  with  subject  matter  expert  assistance.  Analysts  werc 
asked  to  ensure  that  the  instructional  characteristics  listing  was  complete  in 
that:  (1)  if  a  characteristic  was  not  on  the  list  and  seemed  appropriate, 
it  would  be  added,  or  (2)  if  characteristics  as  they  stood  did  not  adequately 
distinguish  the  requirement,  such  a  characteristic  would  be  redefined.  A  few 
changes  did  occur.  For  example,  visual  and  aural  data  displays  were  added  to 
the  list,  while  some  characteristics  were  considered  to  be  redundant  and  elimin¬ 
ated  during  the  course  of  analysis.  Such  activities  demonstrate  the  flexibility 
of  AIMS  and  its  capability  to  maximize  the  information  utilized. 

MATCHING  AND  TRADE-OFFS 

Each  objective  was  analyzed  within  four  mission  phases:  Search,  Analysis, 
Classification,  and  Local ization.  Table  6  summarizes  the  rankings  of  the  six 
alternatives  for  each  of  the  four  mission  phases.  Several  insights  into  the 
media  selection  process  were  gleaned  from  these  data.  During  earlier  phases  of 
the  project  there  was  a  growing  concern  over  whether  or  not  an  expensive 
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coustic  sound  generating  module("e"  in  the  list  above  and  Alternative  5  in 
able  6)would  be  required  for  deployed  training.  This  module,  as  well  as  the 
less-expensive,  on-board  magnetic  tape  system("b"  in  the  list  above  and  Alter¬ 
native  2  in  Table  6)was  included  in  the  media  pool.  Other  questions  related 
to  the  need  for  full  dynamic  control  capability  and  computer-control  led  diagnosis, 
prescription,  feedback,  and  evaluation  were  also  raised.  Therefore,  several 
alternatives  providing  these  capabilities  were  included  in  the  media  pool. 

Table  6.  SUMMARY  OF  DTS  MEDIA  SELECTION  RANKINGS 
Ranking  of  Six  Alternatives  by  Mission  Phase 


Selection 

Ranking 

Search 

Analysis 

Classification 

Local i zation 

FIRST 

Alternative 

5 

Alternative  4 

Alternative  5 

Alternative  5 

SECOND 

Alternative 

3 

Alternative  5 

Alternative  3 

Alternative  3 

THIRD 

Alternative 

4 

Alternative  3 

Alternative  2 

Alternative  2 

FOURTH 

Alternative 

2 

Alternative  6 

Alternative  6 

Alternative  6 

FIFTH 

Alternative 

6 

Alternative  2 

Alternative  4 

Alternative  4 

SIXTH 

Alternative 

I 

At  temati  ve  1 

Alternative  1 

Alternative  1  . 

After  the  definition  of  the  media  pool,  it  was  decided  by  the  sponsoring 
activity  that  a  field  tryout  of  the  tape-based  system  was  needed  to  determine 
its  effectiveness.  The  conclusions  drawn  from  the  media  selection  rankings 

summarized  in  Table  6  confirm  this  decision.  Although  the  tape  system,  with 
instructional  material  support  did  not  rank  consistently  as  high  across  the 
objectives  as  the  acoustic  module,  it  did  satisfy  the  training  requirements, 
as  judged  by  the  media  selection  personnel,  and  would  be  much  less  expensive 
The  final  determination  of  the  tape  system  effectiveness,  however,  should  be 
based  on  an  evaluation  in  the  real  training  environment  since  little  is 
known  of  reliability  and  other  important  factors. 
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SECTION  V 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  design  of  an  automated  aid  to  media  selection  was  based  on  a  common 
logic  found  in  each  of  the  existing  media  selection  models  reviewed  and  an 
analysis  of  functions  which  are  performed  by  media  selection  analysts.  That 
common  logic  is  the  matching  of  instructional  attributes  required  by  the 
training  to  attributes  possessed  by  the  available  media.  The  AIMS  programs 
were  designed  .0  provide  an  efficient,  flexible  computer-based  system  for 
media  selection.  The  user  has  total  flexibility  in  defining  a  model  to  address 
a  particular  problem. 

CONCLUSIONS 

Computer-based  automated  aids  for  instructional  system  development  snould 
provide  the  user  two  advantages:  (1)  reduce  the  labor  intensive  efforts,  and 
(2)  allow  a  more  effective  development  effort  to  be  accomplished.  At  the 
beginning  of  the  project,  five  goals  were  defined  for  AIMS  which  relate  to  these 
advantages.  They  are: 

a.  Maximize  the  use  of  information  pertinent  to  media  selection. 

b.  Provide  flexibility  in  modifying  the  model  to  reflect  specific 
problems  and  changing  technology  of  media  or  media  guidelines. 

c.  Provide  for  consideration  of  student  stage  of  learning. 

d.  Allow  greater,  more  detailed  selection  of  training  devices, such  as 
simulators  or  other  hands-on  devices. 

e.  Allow  for  variations  in  the  stage  of  front-end  analysis. 

These  goals  were  achieved,  in  part,  by  designing  AIMS  to  allow  the  user  to 
define  the  instructional  attributes  required  of  the  media  and  determine  the 
pool  of  media  to  be  considered.  Table  7  summarizes  the  benefits  of  AIMS 
associated  with  each  of  the  five  goals.  The  AIMS  system  assists  the  media 
selection  team  in  defining  a  particular  model  to  meet  specific  needs  or  in  using 
an  existing  model . 

In  addition,  as  described  in  Section  IV,  a  large  gain  is  made  functionally 
toward  solving  the  problems  associated  with  media  selection  in  highly  unique 
training  situations  where  it  is  necessary  to  determine  clearly  the  required 
characteristics  of  a  nonexistent  training  device,  and  to  determine  whether  an 
investment  in  such  equipment  is  justifiable.  Such  problems  are  not  usually 
approachable  with  a  generalized,  cognitivc-learning-based  media  selection 
model . 

The  approach  toward  these  goals  through  a  computerized  aid  to  media 
selection  resulted  in  what  appears  to  be  a  significant  reduction  in  the  labor 
intensive  aspects  of  this  component  cf  ISD.  Experience  with  the  AIMS  system 
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Table  7.  BENEFITS  OF  AIMS 


GOAL 

1.  Maximize  Information  Used 


2.  Provide  Flexibility  in 
Defining  Model  to  Reflect 
Specific  Problems 


3.  Consider  Stage  of 
Learni ng 


4.  Better  Selection  of  Hands- 
On  Training  Devices 


5.  •  Allow  Variations  in  Stage 
of  Front-End  Analysis 


LABOR  AND/OR  DEVELOPMENT  BENEFIT 

User  may  define  problem  as  complex  as 
desired  because  the  automation  will 
considerably  reduce  the  labor  time 
needed  for  matching  and  making  trade¬ 
offs. 

The  user  may  define  a  totally  new  data 
base  of  attributes  and  media  which,  in 
effect,  becomes  a  problem-specific  media 
selection  model.  Changes  to  the 
attributes  or  media  pool  are  easily  made. 

Whatever  the  user  decides  is  an 
appropriate  categorization  of  stage  of 
learning  can  be  defined  in  AIMS  as  an 
instructional  attribute.  Media  are  then 
scored  and  ranked  on  stage  of  learning 
capabilities  as  well  as  other  specific 
attributes  the  user  wishes  to  consider. 

The  user  explicitly  defines  the  stage 
of  learning,  fidelity  and  task 
characteristic  variables  which 
differentiate  types  and  amounts  of 
hands-on  training  needed. 

The  user  may  define  the  media  pool  and 
attributes  in  as  detailed  a  level  as  is 
appropriate  to  the  information  available, 
from  gross  to  fine.  The  judgments  on 
which  instructional  attributes  are 
required  may  be  done  with  a  mission 
analysis,  hardware  functional  descrip¬ 
tion,  task  analysis,  or  learning 
objectives.  AIMS  will  save  time  in 
defining  the  pools  and  attributes,  and 
performing  trade-off  analyses. 
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has  shown  that  automating  even  a  small  media  pool  (e.g.,  20  media  by  20 
attributes)  speeds  up  the  selection  process  considerably.  In  a  simple  test, 
media  selections  from  the  media  pool  were  performed  manually  for  12  objectives. 

The  overall  time  for  manual ly  matching  the  attributes  for  12  objectives  to 
appropriate  media  was  41.5  minutes,  or  3.45  minutes  per  objective.  The  same 
12  objectives  assessed  using  the  AIMS  system  required  only  9.40  minutes,  or 
0.78  minutes  per  objective.  If  these  figures  are  projected  to  reflect  a  major 
training  system  with  1000  objectives,  the  time  savings  gain  is  over  a  man-week. 
Furthermore,  if  AIMS  were  used  to  vary  attributes  or  media  pool  items  to  make 
trade-offs  and  run  several  times,  the  savings  could  extend  into  several  man- 
weeks  or  even  months.  It  is  not  uncommon  for  Navy  training  systems  to  have 
several  thousand  objectives  and  the  implications  for  time  savings  and  training 
quality  benefits  are  clear. 

RECOMMENDATIONS 

The  power  of  AIMS  to  allow  the  user  to  develop  specific  media  selection 
models  to  reflect  specific  problems,  suggested  several  possible  efforts  for 
additional  use  of  AIMS.  First,  AIMS  could  be  used  to  build  an  evaluation  tool 
for  new  delivery  systems.  For  example,  a  general  set  of  attributes  which 
reflects  a  broad  set  of  Navy  traininq  objectives  could  be  identified  to  match 
against  a  standard  media  pool.  As  a  new  delivery  system  is  introduced  for 
consideration, its  ability  to  perform  on  the  training  attributes  could  be  added 
to  the  data  base.  The  new  media  could  then  be  compared  against  others  on  the 
data  base. 

A  related  use  of  AIMS  could  be  in  evaluacino  media  of  a  particular  kind. 

For  example,  media  pools  consisting  of  only  videodisc  systems,  or  all  CAI 
systems,  or  all  slide/sound  systems  could  be  developed  and  rankings  determined 
based  on  selected  learning  objective  clusters  of  interest.  A  logical  extension 
of  this  concept  would  be  to  build  a  media  pool  consisting  of  design  alternatives 
for  a  particular  training  capability  which  needs  to  be  developed.  This  is 
an  application  frequently  needed  in  which  AIMS  can  be  a  valuable  tool. 

The  above  uses  of  AIMS  for  delivery  system  evaluation  could  provide  a  tool 
which  does  not  exist  now.  The  AIMS  system  would  not  provide  final  data  for 
answers  to  evaluation  questions,  but  it  would  provide  a  systematic  approach 
to  the  process  and  give  support  for  decisions  made.  It  is  recommended  that  these 
types  of  AIMS  applications  be  explored  by  further  study. 

A  second  application  area  centers  on  the  cost  analysis  of  media..  In  the 
current  study,  the  analysis  of  media  training  effectiveness  was  central  and  that 
is  what  AIMS  is  intended  to  help  with.  However,  no  media  selection  can  be 
complete  without  cost  analysis.  The  current  TECEP  costing  technique  already 
provides  a  cost  model  which  will  work  for  most  Navy  efforts.  It  is  recommended 
that  the  TECEP  cost  model,  or  another  if  appropriate,  be  interfaced  to  AIMS  so 
that  all  decision  parameters  can  be  guided  by  the  analyst  with  computer  assistance. 
Cost  data  for  acquisition  and  operation  of  media  should  be  included  to  provide 
comparisons  for  selecting  media,  but  not  for  providing  budget  estimates. 

Budget  estimates  are  appropriate  for  only  a  small  set  of  alternatives  where  a 
full  economic  analysis  should  be  performed. 
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SECTION  I 
INTRODUCTION 


Selecting  the  most  effective  and  efficient  media  for  a  training  program 
is  a  difficult  task.  For  each  training  objective,  there  are  specific 
instructional  cha  racteri  sti  cs  and  requirements  which  must  be  met.  And  for 
each  individual  set  of  instructional  requirements,  there  are  usually  several 
media  available  that  meet  those  requirements.  These  media,  in  turn,  vary 
both  generally  in  their  ability  to  deliver  instruction  and  specifically  in 
their  ability  to  meet  the  exact  requi  rements  of  a  given  objective. 

In  the  past,  media  selection  models  have  tended  to  group  training 
objectives  into  broad,  general  media  categories  such  as  "CAI,"  "workbook"  or 
"simulator."  Beyond  this  level,  tnere  were  few  systematic  techniques  for 
further  specifying  the  exact  media  to  employ  or  to  purchase.  The  media  pools 
from  which  the  models  made  their  selections  were  rigid  and  confined.  Few 
models  were  designed  to  be  adaptable  to  meet  the  needs  of  a  variety  of 
training  system  development  programs,  and  few  could  be  updated  to  change  with 
emerging  weapon  systems  or  changing  media  technologies.  The  Automated 
Instructional  Media  Selection  (AIMS)  model  was  designed  to  overcome  most  of 
the  earlier  shortcomi ngs .  The  system  provides  a  general  media  pool  from 

which  to  make  selections,  but  this  pod  is  completely  editable  and  updatable. 
In  fact,  the  user  of  the  system  can  develop  a  completely  unique  media  pool, 
specifically  tailored  to  the  needs  of  a  given  training  program  and  input  the 
pool  into  the  computer  as  a  data  case  without  the  assistance  of  computer 
specialists  or  programming  personnel. 

The  system  is  designed  to  allow  for  an  efficient  division  of  labor  among 
personnel.  Media  specialists  can  focus  on  developing  media  pools,  listing  up 
to  99  potential  media  from  wnich  to  select,  and  up  to  99  media  attributes. 
Each  medium's  ability  to  deliver  each  attribute  can  be  weighted  independently 
by  the  media  experts  to  then  allow  the  computer  to  make  final  discriminations 
among  the  selected  media.  Educational  specialists  and  subject  matter  experts 
can  focus  on  determining  tne  instr.u  ional  characteristics  necessary  to  meet 
each  training  objective.  A  simple  checklist  is  provided  to  simplify  and 
facilitate  this  task.  Clerical  personnel  can  enter  all  data  into  the 
computer,  and  the  computer,  in  turn,  assesses,  selects,  calculates,  ranks, 
prints  out,  and  records  the  most  appropriate  media  for  each  objective  within 
a  matter  of  seconds. 

This  manual  is  designed  to  provide  general  instructions  for  using  the 
AIMS  programs.  It  is  not  intended  to  provide  a  course  in  instructional 
design  or  media  selection.  Even  if  you  are  quite  unfamiliar  with  computers, 
stepping  through  each  program  on-line  with  the  computer,  using  this  manual  as 
a  guide,  will  provide  enough  knowledge  and  practice  to  allow  you  to  use  the 
system  without  the  need  for  special  programming  skills. 

Good  luck.  You  can  expect  to  make  some  mistakes  when  first 
familiarizing  yourself  with  the  system,  but  the  computer  will  help  by 
catching  some  mistakes  and  sounding  a  tone  when  they  occur.  As  you  gain 
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experience  with  the  system  and  better  understand  its  capabilities,  the 
possibilities  and  potential  grow  as  far  as  your  imagination  will  let  them, 
and  the  work  load  of  the  media  selection  process  lessens. 
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USING  THE  AIMS  SYSTEM 

The  Automated  Instructional  Media  Selection  (AIMS)  model  is  designed  to 
provide  rapid  access  to  a  data  base  of  up  to  99  media  and  up  to  99 
instructional  cha  racteri  sties .  Tne  basic  procedures  of  the  system  require 
that  the  persons  responsible  for  designing  and  developing  instruction 
determine  the  instructional  characteristics  and  media  attributes  necessary  to 
achieve  a  given  training  objective.  Once  these  requirements  are  determined, 
the  data  are  entered  into  the  computer.  The  computer  then  selects  those 
media  which  best  match  the  instructional  requirements  of  the  training 
objective.  The  selected  media  are  ranked  in  order  of  appropriateness,  and 
the  number  of  times  each  medium  is  selected  is  tabulated  and  printed  out  in  a 
report . 


The  system  contains  many  instructional  passages  and  menus  that  appear 
throughout  the  program  tc  make  it  self-explanatory,  but  a  review  of  this 
manual  is  recommended  before  attempting  to  do  any  serious  work  on  the  system. 
Initially,  you  should  step  through  each  phase  of  the  AIMS  programs  with  this 
manual  as  a  guide.  Some  of  the  steps  in  the  procedure  are  off-line.  That 
is,  they  take  place  on  paper  without  the  use  of  the  computer.  Other  steps 

are  on-line,  and  require  direct  interaction  with  the  computer.  In  general, 

the  computer  does  all  of  the  tasks  requiring  "rote  labor"  (such  as  checking 
the  appropriateness  of  various  media  for  objectives,  calculating  rankings, 
printing  and  filing  information),  while  a  knowledgeable  human  being  is  still 
required  for  those  judgmental  tasks  requiring  experience  and  expertise  (such 
as  determining  the  necessary  instructional  characteristics  of  a  given 
obj ecti ve) . 

Although  the  user  does  not  need  to  know  anything  about  computer  science 
or  computer  programing  to  use  the  AIMS  system,  typically  a  person  with  some- 
computer  experience  is  requi  red  to  get  the  system  up  and  running  on  a  given 
computer.  A  programmer's  guide  with  the  information  necessary  to  perform  that 
task  is  available.  However,  as  the  user,  you  can  begin  learning  to  use  the 
system  by  following  the  six  steps  below. 

These  steps  describe  how  to  operate  the  AIMS  system  using  the  standard 
AIMS  media  pool.  However  one  of  the  strengths  of  the  AIMS  system  is  that  the 
user  can  edit  any  aspect  of  the  media  pool  at  any  time  to  make  it  more 
accurately  fit  the  characteristics  of  the  training  program  or-  more 
real istical ly  reflect  the  readily  available  media  alternatives.  The  user  can 
even  create  an  entirely  new  media  pool  without  disturbing  the  standard  AIMS 
media  pool.  If  you  want  to  update  or  refine  the  standard  AIMS  media  pool  or 
create  your  own  media  pool,  refer  to  Sections  IV,  V,  and  VI  of  this  manual 
before  completing  Steps  3  through  6  below. 
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STEP  1:  GETTING  THE  SYSTEM  RUNNING  O'  THE  COMPUTER  (ON-LINE) 

The  first  step  is  to  turn  on  the  computer  and  the  printer.  Then, 
typically,  you  will  want  to  insert  the  AIMS  disk  into  Drive  A  (or  Drive  I) 
and,  if  your  system  has  a  second  disk  drive,  you  will  want  to  insert  a  data 
disk  into  it.  If  you  are  already  confused,  consult  someone  vrtio  knows 
something  about  your  particular  computer  system  and  how  the  AIMS  system  is 
set  up  on  it.  Then,  when  you  are  "booted  up,"  you  will  want  to  type  RUN 
AIMS  followed  by  a  <RETURN>*  or  'ENTER1.  If  that  does  not  work,  try  typing 
CRUN2  AIMS  followed  by  a  <RETURN>  or  'ENTER'.  The  AIMS  menu  should  appear  on 
the  screen. 

STEP  2:  THE  AIMS  MENU  (ON-LINE) 

When  the  menu  appears  on  the  screen,  it  will  read  as  follows.  .  . 

WELCOME  TO  AIMS 

The  Automated  Instructional  Media  Selection  (AIMS)  model  is  designed  to 
select  media  from  either  a  standardized  media  pool,  called  AIMS,  or  from 
a  user-defined  media  pool.  The  first  step,  here,  is  to  type  in  the  name 
of  the  media  pool  you  will  be  using.  If  you  want  to  use  the  standard 
AIMS  pool,  just  type  in  AIMS  followed  by  a  <RETURN>.  If  you  are  going 
to  be  creating  a  new  media  pool,  type  in  an  original  name  (no  more  than 
£  characters,  please,  and  no  blank  spaces  or  punctuation  marks). 

ENTER  THE  NAME  OF  THE  MEDIA  POOL  YOU  WILL  BE  USING  : 

The  following  steps  assume  you  will  be  using  the  standard  media  pool  for  the 
AIMS  system.  Its  name  is  AIMS,  so  type  in  AIMS  <RETURN>.  If  the  computer 
comes  back  and  says  .  .  . 

THAT  FILE  DOES  NOT  EXIST  :  DO  YOU  WISH  TO  CREATE  A  NEW  MEDIA  POOL 
WITH  THE  NAME  'AIMS'  ?  (Y/N) 

type  in  an  N  <RETURN>  and  consult  your  programmer.  Otherwise  you  should  see 
the  following  message  .  .  . 

YOU  CAN  OPT  TO  DO  ANY  OF  THE  SEVERAL  FUNCTIONS  BELOW  AT  THIS  POINT. 

EACH  OF  THESE  FUNCTIONS  IS  DESCRIBED  IN  DETAIL  IN  THE  USER'S  MANUAL. 

SELECT  THE  FUNCTION  YOU  WISH  TO  PERFORM  FROM  THE  CHOICES  BELOW: 

1.  SELECT  MEDIA 

2.  CREATE  A  MEDIA  POOL  DATA  BASE 

3.  EDIT  A  MEDIA  POOL 

4.  PRINT  MEDIA  POOL/SELECTION  INFORMATION 

5.  RETURN  TO  OPERATING  SYSTEM 


*”The  symbol  <RETURN>  refers  to  depressing  the  carriage  return  key  (called  the 
'ENTER'  key  on  some  computers).  All  user  inputs  are  generally  followed  by  a 
<RETURN>. 
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If  you  wish  to  create  a  unique  media  pool  or  change  the  standard  AIMS 
media  pool,  you  should  select  function  2  or  3  from  the  AIMS  menu  before  going 
on  to  Step  3. 

STEP  3:  PRINTING  MEDIA  POOL  INFORMATION  (ON-LINE) 

Before  you  actually  select  media  (function  1  from  the  AIMS  menu),  you 
will  need  information  about  the  AIMS  media  pool.  This  is  a  simple  process. 

Select  function  4  PRINT  MEDIA  POOL/SELECTION  INFORMATION  from  the  START 
menu  by  typing  in  a  4  <RETl)RN>.  The  PRINT  menu  will  then  appear  on  the 
screen.  Refer  to  Section  III  of  this  manual  for  detailed  descriptions  and 
examples  of  the  print  functions. 

STEP  4:  MATCHING  OBJECTIVES  TO  INSTRUCTIONAL  ATTRIBUTES  (OFF-LINE) 

The  next  step  in  the  media  selection  procedure  takes  place  without  the 
aid  of  the  computer.  This  is  one  of  those  human,  judgmental  tasks  that  the 
computer  cannot  perform.  The  procedure  is  as  follows.  First,  you  need  to 
obtain  a  list  of  the  objectives  (or  task  statements)  for  \Miich  you  are 
interested  in  selecting  media.  Next,  you  need  to  make  at  least  one 
selection  worksheet  for  each  objective  on  that  list  either  by  printing  the 
worksheet  repeatedly  with  the  computer  or  by  photocopying  the  original. 
Then,  stepping  through  one  objective  at  a  time,  fill  in  the  objective  number 
on  a  worksheet.  You  can  jot  down  the  objective  in  the  space  provided  on  the 
worksheet,  if  you  so  desire.  Finally,  check  off  the  boxes  on  the  worksheet 
that  represent  the  instructional  characteristics  required  to  meet  the 
objective  under  consideration. 

Figure  A-l  shows  a  completed  worksheet  for  hypothetical  objective  number 
1.1.2.,  STATE  INSPECTION  POINTS  OF  LIFE  VEST.  As  showr.,  the  person 
determining  the  necessary  instructional  attributes  for  this  objective  felt  it 
should  be  classified  as  (1)  Cognitive  learning  type  with  display 
characteristics  of  (4)  Verbal /Text  and  (6)  Drawings  as  sufficient 
instructional  attributes.  Hence,  the  appropriate  boxes  are  checked  off  on  the 
worksheet . 

As  another  example,  the  objective  PERFORM  EMERGENCY  PROCEDURES  FOR 
NAVIGATION  EQUIPMENT  FAILURE  would  probably  be  classified  as  Psychomotor 
learning  type  and  require  more  extensive  instructional  characteristics,  such 
as  Kinesthetic  Cues,  Immediate  Feedback,  Manipulative  Response  and  Grew/Team 
Interaction.  If  this  were  the  case,  then  the  appropriate  boxes  would  be 
checked  off  on  the  worksheet  for  that  objective  and  the  computer  would  make 
selections  accordingly. 

In  general,  it  is  best  to  perform  this  step  for  a  group  of  objectives 
before  going  back  on-line  with  the  computer.  There  will  often  be 

disagreements  among  raters  as  to  what  are  essential  characteristics,  but  this 
is  a  human,  judgmental  task  that  the  computer  cannot  aid  in.  Once  a  number 
of  worksheets  have  been  completed,  you  can  go  back  on-line  to  begin  the  next 
step. 


73 


NAVTRAEQUIPCEN  79-C-0104-1 


ELECTION  WORKSHEET  FOR  USE  WITH  THE  FILE  :  AIMS 


Objective  Number: 


/.  l.^L 


Obj  ecti  ve: 


Put  a  check  in  the  boxes  next  to  the  attributes  requi  red  of 
any  medium  which  might  be  used  to  meet  this  objective. 


*  I.  TYPE  OF  LEARNING 

1.  Cognitive 
f  f  2.  Psychomotor 

f  f  3.  Affective  (atti  tudi  nal ) 

*  1 1. "Display  characteristics 

4.  Verbal /Text 
f _ 1  5.  Photographs 

6.  Drawings 
II  f  7.  Diagrams 

f _ f  8.  Colors 

If  9.  Visual  Motion  (constant) 
f  f  10.  Visual  Motion  (variable 

f _ f  11.  3-Dimensional 

f _ 1  12.  Audio  (voice) 

f  1  13.  Audio  (ambient) 

f _ f  14.  Tactile  Cues 

f  f  15.  Kinesthetic  Cues 

*  IIimESPONSL  MODE 

f _ 1  lo.  Verbal /written  response 

f  1  17.  Voice  response 

f  f  18.  Decision  Indicator 

f  f  19.  Fine  Motor  Response 

f  f  20.  Gross  Motor  Response 

f  f  21.  Tracking  Response 

1 _ f  22.  Manipulative  Response 


1  1  23.  Continuous  Response 

IV. “EVALUATION  MODE 

1 _ 1  24.  Instructor  Evaluati on 

f _ 1  25.  Self  Evaluation 

f _ f  26.  Peer  Evaluation 

f  1  2  7.  Automated  Evaluation 

1 _ f  2  8.  Verbal /Written  Eval. 

f  1  29.  Immediate  on  Response 

f  f  30.  Immediate  on  Error 

f  1  31.  Post-Session  Evaluation 

v.  Environmental  conditions 

f _ 1  32.  Physical  Motion 

f  f  33.  Acceleration 

f~f  34.  Pitch /Roll 
1  l  35.  Vibration 

f  1  36.  Restrictive  Clothing 

VI. TEARNING  LEVELS 

f _ f  37.  a.  Familiarity 

f  f  3  6.  b.  Perform  Procedure 

f  f  39.  c.  Perform  Job 

1  f  40.  d.  Perform  Mi ssi on 

VII.  SPECIAL  REQUIREMENTS' 

1 _ f  41.  Crew/Team  Interaction 

f  f  42.  Large  Memorization  Comp 

f _ 1  43.  Random  Access  Logic 


Figure  A-l.  Example  of  completed  media  selection  worksheet  using 
the  standard  AIMS  medio  pool. 
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STEP  5:  SELECTING  MEDIA  (ON-LINE) 

If  you  disconnect  yourself  from  the  computer  to  perform  the  previous 
step,  you  will  have  to  get  the  AIMS  system  up  and  running  on  the  computer 
again.  Then  you  will  need  to  key  in  the  appropriate  file  name  (AIMS)  and 
selec„  function  1,  SELECT  MEDIA,  from  the  START  menu.  If  you  are  still  on¬ 
line,  just  type  in  a  1  <RETURN>.  You  will  get  the  following  message  on  the 
CRT  .  .  . 

TO  USE  THE  MEDIA  POOL,  AIMS,  MAKE  SURE  THAT  THIS  FILENAME  APPEARS  AT 
THE  TOP  OF  THE  WORKSHEET  THAT  YOU  WILL  BE  KEYING  IN  THE  DATA  FROM.  THEN 
TYPE  IN  EACH  OF  THE  NUMBERS  THAT  HAVE  BEEN  CHECKED  OFF  ON  THE  WORKSHEET, 
BEING  SURE  TO  FOLLOW  EACH  NUMBER  WITH  A  <RETURN>. 

FOR  EXAMPLE,  IF  BOXES  1,  3,  9,  AND  14  ARE  CHECKED  OFF  ON  THE  WORKSHEET, 
YOU  WANT  TO  TYPE  IN: 

1  <RETURN> 

3  <RETURN> 

9  <RETURN> 

14  <RETURN> 

0  <RETURN> 


WHEN  YOU  ARE  DONE,  TYPE  IN  A  'O'  AS  THE  LAST  NUMBER. 

IF  YOU  NEED  MORE  INFORMATION,  REFER  TO  THE  USER’S  MANUAL. 


STRIKE  ANY  KEY  (FOLLOWED  BY  <RETURN>  )WHEN  READY  TO  CONTINUE  : 


Going  back  to  the  first  example,  you  want  the  computer  to  select  media 
with  attributes  (1)  Cognitive  learning  type,  (4)  Verbal/Text  and  (6)  Drawings 
for  objective  1.1.2.  First,  strike  a  key  and  hit  <RETURN>.  You  will  get  the 
following  message.  .  . 

DO  YOU  WANT  TO  PRINT  OUT  SELECTION  MEDIA  ON  PAPER  (Y/N)  ?  : 

If  you  are  hooked  up  to  a  printer,  type  in  a  Y  <RETURN>,  otherwise  type  in  a 
N  <RETURN>.  Then  the  message  .  .  . 

WHAT  OBJECTIVE  NUMBER  DO  YOU  WANT  TO  ACCESS  ?  : 

appears  on  the  screen.  At  this  point,  simply  key  the  objective  number  from 
the  worksheet  into  the  computer,  followed  by  a  <RETURN>.  (In  the  present 
example,  you  would  type  in  1.1.2  <RETURN>.)  The  next  message  to  appear  on 
the  screen  is  .  .  . 

ENTER  AN  ATTRIBUTE  NUMBER  (or  0  if  done)  : 

Referring  to  the  worksheet  for  the  example  (Figure  A-l),  you  see  that  1  is 
the  first  box  checked  off,  so  you  would  key  in  a  1  <RETURN>.  The  very  same 
message  appears  on  the  screen  agjin.  The  screen  looks  like  this  now  .  .  . 
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ENTER  AN  ATTRIBUTE  NUMBER  (or  0  if  done)  :  1 
ENTER  AN  ATTRIBUTE  NUMBER  (or  0  if  done)  : 

The  computer  is  expecting  another  attribute  number.  The  next  one  checked  off 
on  the  worksheet  for  the  example  is  a  4.  If  you  key  this  number  in,  you  get 
the  same  message  again.  Only  this  time  the  screen  looks  like  this  .  .  . 

ENTER  AN  ATTRIBUTE  NUMBER  (or  0  if  done)  :  1 
ENTER  AN  ATTRIBUTE  NUMBER  (or  0  if  done)  :  4 
ENTER  AN  ATTRIBUTE  NUMBER  (or  0  if  done)  : 


The  next  attribute  number  in  the  example  is  6,  so  you  would  key  in  a  6 
<RETURN>.  Since  you  do  not  have  any  more  attributes  to  key  in,  you  would 
type  in  a  0  <RETURN>.  At  this  point,  the  computer  begins  its  work.  It  looks 
at  each  medium  in  the  media  pool  and  selects  those  media  vrfiich  have  the 

attributes  you  have  entered.  Next,  it  looks  at  how  those  media  have  been 

rated  in  relation  to  the  selected  attributes  and  decides  *hich  is  the  best 
medium  for  these  attributes,  the  second  best  medium,  third  best  medium,  etc. 
Then,  it  prints  out  the  results  on  either  the  CRT  or  the  printer  (depending 
on  how  you  responded  to  the  previous  hardcopy  question). 

Figure  A-2  is  an  example  of  the  printout  provided  by  the  computer 
describing  its  selection  of  media  for  objective  number  1.1.2.  As  shown,  the 
media  are  printed  out  in  rank-order  fashion  based  on  the  numbers  in  the 
specific  rating  column.  This  specific  rating  is  the  computer's  calculation 
of  how  well  the  selected  medium  meets  the  specific  instructional 
characteristics  of  this  objective.  Note  that  several  media  were  selected, 

with  Workbook  as  the  primary  choice.  Generally  you  will  not  be  too  concerned 

about  the  media  selected  for  a  single  objective,  since  it  is  media 
information  about  groups  of  objectives  that  is  important  in  the  selection 
process.  The  AIMS  system  provides  information  about  media  selections  for 
groups  of  objectives  in  Step  6.  For  now,  you  need  to  keep  keying  in  data 
about  the  instructional  cha racteri sti cs  of  the  objectives.  To  do  so,  answer 
the  computer's  next  question  .  .  . 

ARE  YOU  READY  FOR  THE  NEXT  OBJECTIVE  (Y/N)  : 

by  keying  in  a  Y  <RETURN>.  The  computer  will  repeat  the  process  just 
described  by  first  requesting  the  objective  number,  then  the  specific 
instructional  characteristics  of  that  objective.  Repeat  this  process  until 
all  of  the  objectives  for  a  given  group  have  been  "mediated."  Then,  when  the 
computer  asks  if  you  are  ready  for  the  next  objective  type  in  an  N  <RETURN> 
and  the  computer  will  print  out  a  summary  of  the  media  selections  made  for  a 
given  group  of  objectives. 

Figure  A-3  presents  an  example  of  a  final  tabulation  summary.  The  media 
are  presented  in  order  of  most-  to-least  selected,  with  non-selected  media 
not  listed.  The  report  indicates  55  objectives  were  input  in  this  group.  The 
next  two  columns  describe  how  many  times  each  medium  was  selected  and  the 
percent  of  objectives  vrfiich  were  satisfied  by  this  medium,  respectively. 
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MEDIA  SELECTION  FOR  OBJECTIVE  1.1.2 


SPECIFIC 

GENERAL 

MEDIUM 

RATING 

RATING 

1. 

Workbook 

'  3.750 

3.5 

2. 

Random  Access  Slides 

3.250 

3.15 

3. 

Lecture 

3.250 

3.217 

4. 

Motion  Picture 

2.750 

2.047 

5. 

Videodisc 

2.500 

3.142 

6. 

Syncro-Pulse  (Cue-See) 

2.500 

2.952 

7. 

Slides 

2.500 

2.937 

9. 

Video  Tape 

2.250 

3.012 

Figure  A-2.  Example  of  media  selection  printout 
using  the  standard  AIMS  media  pool. 
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MEDIA 

SELECTION : 

#  OF 

OBJECTIVES 

FINAL  TABULATION 

%  OF 

OBJECTIVES 

FOR  55  OBJECTIVES: 

MEDIUM 

I  . 

44 

0.800 

OP.  Equip ./OP. Env ironment 

2. 

37 

0.57  3 

Team  Trainer 

3  . 

33 

0.500 

Part  Task  Trainer 

4  . 

25 

0.455 

Demonstration 

5. 

22 

0.  400 

GETS 

6  . 

22 

0. 400 

CAI 

7. 

20 

0.  364 

PLATO  +  Videodisc 

S  . 

19 

0.  345 

SI  ides/ Aud io 

9. 

19 

0.  345 

Motion  Picture/Videotape 

10. 

17 

C  .  309 

Besseler  Cue-See 

i : . 

17 

0.309 

TICCIT  +  Videodisc 

12. 

15 

0  .  273 

Lecture 

13. 

12 

C.  218 

3-D  Mock-up 

'  A 
*1  • 

12 

0.218 

OP.  Equip. /Lab 

15. 

12 

0.  218 

Teaching  Machine 

15. 

11 

0.  200 

Slides 

17. 

1 1 

0. 200 

Audio  Tape 

18. 

10 

0.  187 

Procedural  Trainer 

19. 

10 

0.  182 

Random  Access  Slides 

20  . 

9 

0.164 

Wo  r  kboo  k 

21  . 

8 

0.  145 

Group  Discussion 

22. 

5 

0.091 

2-D  Mock-up 

Figure  A-5.  Example  of  final  tabulation  summary 

for  media  selected  for  a  group  of  objectives. 
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The  name  of  the  medium  is  presented  in  the  last  column.  So  in  the  example  in 
Figure  A-3,  the  medium  OP.  Equipment  was  selected  44  times  (out  of  55 
possible).  It  satisfied  80  percent  of  the  55  objectives.  Medium  number  22 
on  the  list,  2-D  Mock-up,  was  selected  only  five  times,  satisfying  only  9 
percent  of  the  objectives. 

The  final  tabulation  summary  is  valuable  in  providing  an  overview  of  the 
media  most  applicable  to  a  given  set  of  objectives.  This  is  particularly 
important  during  very  early  stages  of  training  systems  development  vrfien  a 
general  overview  of  eventual  media  needs  is  all  that  can  be  expected.  The 
rank  order  of  the  media  also  gives  some  idea  of  the  criticality  of  each 
medium  for  that  set  of  objectives. 

However,  as  described  in  Step  6  below,  more  information  is  available 
about  the  computer's  selection  of  media  than  the  summary  provides.  The 
computer  also  keeps  a  record  of  each  objective  number  and  the  five  highest- 
ranked  media  selected  for  each  objective,  as  well  as  the  specific  rating 
assigned  each  medium.  Depending  on  how  your  programmer  sets  up  the  system, 
you  will  probably  have  different  data  bases  of  objectives  with  their  selected 
media  on  separate  disks. 

STEP  6:  PRINTING  SELECTION  DATA  (ON-LINE) 

When  all  of  the  objectives  in  a  given  group  have  been  through  the 
selection  process,  you  will  probably  want  information  about  the  media 
characteristics  and  media  combinations.  The  AIMS  system  provides  this 
through  function  4,  PRINT  MEDIA  POOL/SELECTION  INFORMATION,  on  the  START 
menu.  When  you  select  this  function,  you  will  get  the  PRINT  menu.  When  you 
select  function  5,  PRINT  SELECTION  DATA,  from  the  PRINT  menu  you  get  the 
following  series  of  events. 

First,  the  computer  informs  you  that  .  .  . 


THIS  PROGRAM  ALLOWS  YOU  TO  SEARCH  OUT  OBJECTIVES  WITH 
A  GIVEN  MEDIUM  OR  MEDIA  COMBINATION 

then  asks  .  .  . 

DO  YOU  WISH  TO  LIST  OUT  THOSE  OBJECTIVES  FOR 
WHICH  NO  MEDIA  WERE  SELECTED  (Y/N)  : 

If  you  type  in  a  Y  <RETURN>  In  response  to  the  question,  the  computer  will 
ask  .  .  . 


DO  YOU  WANT  A  HARD  COPY  OF  THE  DATA  (Y/N): 

If  you  type  in  a  Y  <RETURN>,  the  computer  will  print  on  paper  those  objective 
numbers  for  which  no  media  were  selected.  Otherwise  it  will  list  those 
objective  numbers  on  the  screen.  The  computer  then  displays  a  new  message  .  . 
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THIS  PROGRAM  ALLOWS  YOU  TO  KEY  IN  UP  TO  5  MEDIA,  AND  THEN 
SEARCHES  THE  DATA  FILE  FOR  OBJECTIVES  WHICH  HAVE  THOSE  MEDIA. 

YO <J  CAN  ENTER  ANY  COMBINATION  OF  MEDIA  THAT  YOU  WISH. 

■Then  the  computer  asks  .  .  . 

ENTER  THE  FIRST  MEDIUM  NUMBER  (0  if  done)  : 

At  this  point,  you  need  to  have  a  list  of  media  on  the  data  base.  Then  you 
can  search  for  any  medium  or  combination  of  media  among  the  objectives  by 
eyeing  in  tne  media  you  are  interested  in.  For  example,  if  you  want  to  pull 
out  .l1  of  the  objectives  for  which  botn  Medium  1  and  Medium  2  were  selected, 
you  would  type  in  a  1  <RETURN>.  Then  the  CRT  would  look  like  this  .  .  . 

ENTER  THE  FIRST  MEDIUM  NUMBER  (0  if  done)  :  1 

ENTER  THE  SECOND  MEDIUM  NUMBER  (0  if  done)  : 

Then  you  would  type  in  a  2  <RETURN>  and  the  screen  would  look  like  this  .  .  . 

ENTER  THE  FIRST  MEDIUM  NUMBER  (0  if  done)  :  1 

ENTER  THE  SECOND  MEDIUM  NUMBER  (0  if  done)  :  2 

ENTER  THE  THIRO  MEDIUM  NUM3ER  (0  if  done)  : 

Since  you  are  only  interested  in  Medium  1  and  Medium  2,  type  in  a  0 
<R£TURN>.  The  computer  then  double  checks  for  accuracy  by  printing  on  the 
CRT.  .  . 


1.  FIRST  MEDIUM  :  1.  WORKBOOK 

2.  SECOND  MEDIUM:  2.  LECTURE 

IS  THIS  OK  ?  (Y/N): 

Since  this  is  v*iat  you  wanted,  type  in  a  Y  <RETURN>.  Then  the  computer 
goes  to  work  finding  all  of  the  objectives  for  which  these  two  media  were 
selected.  Figure  A -4  is  an  example  of  a  report  generated  by  this  program. 
Note  that  up  to  five  media  are  listed  for  each  objective.  These  are  the 
five  highest-ranked  media  selected  for  that  objective  during  the  selection 
procedure.  The  specific  rating  for  each  medium  is  also  included. 

This  function  is  valuable  for  grouping  objectives  into  media  categories, 
which  often  helps  in  determining  howto  organize  lessons.  With  experience, 
the  function  of  searching  the  objectives  for  certain  media  or  combinations  of 
media  becomes  the  most  powerful  aspect  of  the  AIMS  system.  Using  this 
function,  it  is  possible  to  try  out  a  variety  of  media  combinations  without  a 
great  deal  of  e  fort  to  see  how  the  objectives  and  selected  media  relate. 
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SECTION  Hi 

USING  THE  PRINT  FUNCTIONS 

*ve  print  functions  cf  AIMS  re  valuable  for  obtaining  written  records 
*■'  r-o  contents  of  vanojs  files.  Generally,  whenever  a  new  data  base  is 
-••’--c.  you  will  want  tc  use  the  print  functions  to  check  your  accuracy  in 
-v'-'  in  t1-!?  data.  .c  access  the  p-i  rt  functions,  you  need  to  select 
f  0"  4,  3RIn"  MEDIA  POOL /SELECTION  INFORMATION,  from  the  START  menu.  The 
-at  SR'  request  will  oe.  .  . 

*  HA'.'  MEDIA  PjCu  DC  YCu  WANT  TO  ACCESS? 

t-e  name  cf  the  media  pool  you  wish  to  access.  This  will  be  the  media 
eb’  yCu  will  be  accessing  as  long  as  you  are  in  tne  print  funct’c-, 
-.•gi-d'ess  cf  row  many  different  programs  you  use.  To  change  media  poo's, 
?u  u  ve  to  return  to  the  START  menu  and  re-access  the  print  functions. 

-ce  yju  nave  erte-er.  a  valid  file  name,  the  CRT  displays.  .  . 

Print  Menu 

PR.  •/'  uP'IONS  FDR  (your  file  name) 


3.  -IS”  ME  D I A  /  A'  7  R 1 5  _'T  £  S  TABLE 
A1E  SELc.C'l'lDN  wDRNSHEET 

5.  ?RINr  SELECTION  DATA 

6.  RETURN  TO  START 

E\  ■ .. R  SELECrION  <RETURN>: 

Ea cn  time  yo-  select  a  given  print  function,  the  computer  will  ask.  .  . 

DC  YOU  WAN'  A  ?A?£R  COPY  OF  THE  OUTPUT  (Y  nR  N)? 

f  you  nave  a  c-inte-  tnat  i-  -ea  ty  to  print,  type  in  a  Y  <RETGRN>  to  produce 
na-dcopy  of  the  output.  'c-e-wise,  type  in  an  N  <RETUR.N>  and  it  will  just 
■  listed  on  the  CRT.  T- :  ot  -i  ve  functions  are  described  below.  When  you 

y?«  ir  at  <RE'uRN>,  o.  *  ' '  go  out  of  the  print  functions  back  -  to  the 

'ART  menu  . 

•jN>.  .  ION  i :  L.S7  MEDIA 


men  you  select  fu-.;tio-  \  aro  request  a  paper  copv  of  the  output,  a 
■  sting  of  all  the  media  on  the  requested  media  pool  will  be  produced  on  the 
-inter.  Figure  A-5  shows  a  listing  of  the  AIMS  media  pool.  The  name  of  the 


vi  i  pool  is  included  on  the  listing,  so  if  something  seems  amiss  on  your 
*  put  make  sure  you  nave  requested  the  correct  media  pool.  If  not,  return 


-o  tne 


START  menu  and  request  the  print  functions  by  typing  in  a  4  <RETUr..l>. 
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MEDIA  LISTED  ON  THE  FILE  :  AIMS 


1  Workbook 

2  Lecture 

3  Slides 

4  Slide/Tape 

5  \udio  Tape 

6  Random  Access  Slides 

7  Moti  on  Pi  cture 

6  Vi  deo  Tape 

0  Syncro-Pul se  (Cue-See) 

10  Vi  aeodi  sc 

11  Teacning  Machine 

12  2-D  Mock-up 

13  3-D  Mock-up 

14  Procedural  Trainer 

15  Part  Task  Trainer 

16  Team  Trainer 

17  Weapon  Systems  Trainer 

If  High  Fidelity  Simulator 

19  Reali a  (Actual  Equipment) 

20  CAI:  TICCIT 

21  CAI :  PLATO 

22  CM  I /Workbook 


Figure  A-5.  n-.--r.-0ut  of  the  media  listed 

On  the  file  AIMS . 
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FACTION  2:  LIST  ATTRL3UTES 

rij.t'e  A-6  sItows  an  attributes  listing  frcm  the  AIMS  madia  pool .  The 
procedure  for  producing  the  attributes  list  is  identical  to  the  method  for 
listing  madia.  However,  on  the  attributes  listing,  the  headings  preceded  by  art 
asterisk  are  in  Roman  mineral  order.  Tnis  was  accanpl ished  wiien  the  data  base 
was  originally  put  on-line.  Type  in  an  when  typing  in  your  heading, 
fol lowed  bv  the  appropriate  Roman  mineral ,  then  the  attribute  heading.  The 
readings  will  be  stored  exactly  as  you  type  them  in. 

F  - )'  id  I .  oh  Li  ST  MEDIA./  ATTRIBUTES  TABLE 

rig. ire  A- 7  provides  the  media  by  attributes  table  for  the  AIMS  media 
poo. .  Tins  printout  is  a  little  more  complex  than  the  others.  It  displays: 

a.  The  file  (media  pool)  name 

d.  Ail  madia  names  (in  rows) 

c.  Ac tribune  numbers  (across  top  colunn) 

d.  .All  ratings  between  madia  and  attributes  ( irdivid.  .-.1  cells) 

The  attribute  nunbers  vhich  appear  across  the  top  row  are  taken  from  the 
attributes  listing.  Only  tine  nunbers  are  presented  since  it  is  impossible  to 
print  out  attribute  names  to  fit  into  vertical  columns.  The  relationship 
between  the  madia  and  the  attributes  are  rated  cn  a  scale  of  0  (no 
red  itior.sh.ip)  to  5  (strong  relationship)  . 

L-TNCTION  4:  CREATE  SELECTION  WORKSHEET 

The  selection  worksheets  are  produced  by  the  computer  to  allow  trair.irv- 
sys terns  designers  and  developers  to  determine  the  instructional 
c.-iaracten sties  necessary  to  deliver  instruction  for  each  objective.  This 
function  was  described  .in  detail  in  Section  II,  Step  4. 

Function  5:  print  selection  data 

The  selection  data  for  a  giver,  group  of  objectives  is  analyzed  and  Lae:, 
sumarizad  by  the  canoe  --r  to  determine  the  best  medium  or  main 
curb i nations .  This  function  was  described  in  detail  in  Section  II,  Step  6. 
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ATTRIBUTES  LI:  .  C-jL  :  AIMS 

*1.  TYPE  C 

1 .  Cognitive 

2.  Psychomot^. 

3.  Affect'  vt  v-  .. ..  i  .sa  ■  / 

*11.  DISPLAY  S;-A?.A.T  JUST  ICS 

4.  Verbal /lev. 

5.  Photograph 

6.  Drawings 

7.  Diagrams 

8.  Colors 

9.  Visual  V  ,t  •.  ->•  'Stan t) 

10.  Visual  Mi  .  : uD i e) 

11 .  3-Dir.ier.s .  -. 

12.  Audio  { v s • 

13.  Audio  (a  ...  . 

14.  Tactil  T  _ 

15.  Kinestnef: .  .  .  . 

*111.  RESPONSE  •>.. 

16.  Verbal  ,V  *  oi  .c 

17.  Voice  res 

18.  Decision  Irr.: i 

19.  Fine  Me:  ■  • 

20.  Gross  Mote  so 

21.  Tracking  . 

22.  Manipulat; .  a  o^-se 

23.  Continuous  a:.  •  se 

*IV.  EVALUATES  ■  '■>  . 

24.  Instructor  E.s  ;jtion 

25.  Self  Eve.' 

26.  Peer  Eva 

27.  Autorate"  •  '■  n 

28.  Verbal  *•:  -■  . 

29.  Immediate  -  se 

30.  Immediate  :  r-  1 

31.  Post-Sc s 

*V.  ENV1  RCf-M:  ■  -  M  .  '.'IONS 

32.  Physical  '  . 

33.  AcceleraM  ■ 

34.  Pitch/Roil 

35.  Vibrativ- 

36.  Restrict  . 

*vi.  learm*;: 

37.  a.  fa-:' 'I 

38.  b.  Peris. •  ■  -re 

39.  c.  Pertesr. 

40.  d.  Perfo" 

♦VII.  SPt  C •  "  f  lMENTS 

41.  Crew/ u. a.  si  on 

42.  Large  v  .  Component 

43.  Randor  /■--  1 

Figure  A-6.  Printo  4  '  i  u'es  in  the  pool  AIMS. 


This  report  lists  the  media  on  file  AIMS,  and  provides  a  list 
of  numbers  corresponding  to  the  attributes  on  this  file.  The  attributes 
assigned  to  a  given  medium  on  the  data  base  can  be  determined  by  cross-, 
referencing  this  list  of  numbers  with  a  numbered  listing  of  the  attributes 
themsel  ves . 
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SECTION  IV 


EDITING  THE  MEDIA  POOL 

It  is  unlikely  that  a  standard  media  pool  will  be  satisfactory  for  many 
selection  problems.  It  is  also  unlikely  that  a  satisfactory  media  pool  will 
stay  satisfactory  indefinitely.  Therefore,  the  AIMS  system  provides  the  user 
with  several  editing  functions  vtfiich  can  be  used  to  update  or  change  the 
media  pool  data  base  on-line.  These  editing  functions  include:  (a)  adding  a 
medium  to  the  media  pool,  (b)  adding  an  attribute  to  the  media  pool, 

(c) changing  a  single  rating  between  a  given  medium  and  a  given  attribute,  and 

(d)  changing  a  given  medium  or  attribute  and  all  of  the  ratings  that  go  with 
it . 


Section  VI  provides  a  theoretical  discussion  of  how  media  selections  for 
individual  objectives  can  be  used  to  refine  the  selection  process.  The 

mechanics  of  editing  the  media  pool  for  purposes  of  refinement  are  described 

here.  During  this  process,  the  computer  provides  instructions  to  the  user 
and  any  error  that  you  may  make  (other  than  spelling  of  a  medium  or 

attribute)  will  be  caught  by  the  computer  and  you  will  be  able  to  correct  the 
error  quite  easily. 

The  START  menu  allows  you  to  select  function  3,  EDIT  A  MEDIA  POOL.  When 
you  type  in  a  3  <RETURN>,  the  following  menu  appears  on  the  screen.  .  . 

EDIT  MENU 

WHICH  FUNCTION  DO  YOU  WISH  TO  PERFORM  ? 

1.  ADD  A  MEDIUM 

2.  ADD  AN  ATTRIBUTE 

3.  CHANGE  A  RATING 

4.  CHANGE  A  MEDIUM 

5.  CHANGE  AN  ATTRIBUTE 

6.  RETURN  TO  AIMS  MENU 

The  first  five  functions  a-e  described  below.  When  you  type  in  a  6 
<RET'JRN> ,  you  will  go  out  of  the  EDIT  menu  back  to  the  START  menu. 

FUNCTION  1:  ADD  A  MEDIUM 

Changes  in  budgets,  .raining  programs,  and  technology  bring  about  the 
need  for  most  training  systems  designers  to  consider  an  ever-expanding  range 
of  media.  New  media  can  be  added  to  the  media  pool  quite  easily  by  selecting 
function  1  on  the  EDIT  menu.  However,  remember  that  every  new  medium  you  add 
will  have  to  be  cross-referenced  with  all  of  the  attributes  on  the  media 
pool.  It  is  best  to  take  care  of  this  procedure  on  paper  before  working  on¬ 
line.  (You  can  review  this  procedure  in  Section  V  of  the  manual  under  the 
heading  "Cross-reference  media  and  their  attributes.") 
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When  you  type  in  a  1  <RE7URN:%  the  computer  displays  .  .  . 

WHAT  IS  THE  NAME  OF  THE  ML  DUO'  VC  BE  ADDED? 

Then  type  in  the  name  of  the  medium  to  ce  added,  followed  by  a  <RETURN>. 
Next  you  will  be  asked  to  relate  that  medium  to  the  attributes  in  the  media 
pool  using  the  scale  of  0  (no  relationship)  to  5  (strong  relationship) 
discussed  in  Section  V.  The  comoute:  wi1!  automatically  advance  you  through 
the  attributes  and  inform  you  wner.  it  is  done.  At  this  point,  you  will 
return  to  the  START  menu  by  typing  .  6  <RETURN>. 

FUNCTION  2:  ADD  AN  ATTRIBUTE 

This  function  allows  you  tc  aria  new  attributes  to  the  media  pool  data 
base.  The  ability  to  add  attributes  is  very  valuable  in  refining  the 
selection  process,  as  discussed  ir  Sect; cm  VI  of  this  manual.  Like  function 
1,  any  new  attributes  that  you  ac  ’  .v  1 1  have  to  be  cross-referenced  with  all 
the  media  in  the  pool.  This  process^  is  best  performed  off-line  before 
using  the  computer.  (You  can  rev-v.  procedure  in  Section  V  of  tne  manual 
under  the  heading  "Cross-reference  :r -c i ;  and  their  attributes.") 

When  you  type  in  a  2  <RE7URN>  the  computer  displays  .  .  . 

THIS  WILL  BE  AN  ATTRIBUTE  (V  >  >E  -  ILE  (AIMS) 

BEFORE  USING  THIS  PROGRAM.  ;  i\E  YOU 

HAVE  ALL  OF  YOUR  RATINGS  WORKED  OUT  ON  PAPER  FIRST 

TYPE  IN  THE  NEW  ATTRIBUTE  FCLLOWLD  BY  A  <RETURN> 

The  computer  then  requests  thV'  ;r,j  rate  each  medium  with  the  new 
attribute  using  the  O-to-5  seal'-.  p.  completion,  you  will  return  to  the 
START  menu  by  typing  in  a  6  xkETcRh 

FUNCTION  3:  CHANGE  A  RATING 

This  function  allows  you  ir  the  rating  between  any  given  medium 
and  any  given  attribute.  . . ~e  is  simple.  When  you  type  in  a  3 

<RETURN>  from  the  EDIT  menu,  tor  ,  --puter  will  ask  vtfiat  medium  (by  number) 
you  want  to  change  a  rat-ng  or.  T^po  ■  the  number  <RETURN>  and  it  will  ask 
you  what  attribute  (by  number)  you  to  make  the  change  on.  Type  in  the 

attribute  number  <RETURN>  and  tc:  ;  pe  in  the  new  rating  by  number  (0-5) 

<RETURN>. 

FUNCTION  4:  CHANGE  A  MEDIUM 

When  you  select  function  '  f.  r.-  ~.,o  EDIT  menu,  the  computer  displays  .  . 
WHICH  MEDIUM  (by  number)  TO  v'  '  "H  "0  CHANGE?: 
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Tyoe  in  the  number  of  the  medium  you  wish  to  change  <RETURN>.  If  that  medium 
number  does  not  exist,  tne  compute'-  will  tell  you  so  and  ask  you  to  re-enter 
tne  medium  number.  Otherwise  the  computer  will  display  the  medium  requested 
and  ask  you  to  key  in  the  new  medium  <RETURKI>.  The  computer  then  asks  you  if 
you  wish  to  key  in  new  ratings  between  the  new  medium  and  the  present 
.ttributes  on  the  data  base.  If  you  type  in  an  N  <RETURN>,  the  computer  will 
return  to  the  EDIT  menu.  Otherwise  it  will  step  through  each  attribute  and 
■-equest  that  you  rate  the  relationship  between  the  medium  and  attribute 
again.  As  discussed  earlier,  this  procedure  is  best  accomplished  on  paper 
before  working  on-line. 

FUNCTION  5:  CHANGE  AN  ATTRIBUTE 

Like  function  4,  wnen  you  select  function  5  the  computer  will  ask  you  to 
type  in  the  attribute  number  to  be  changed  <RETURN>.  Then  the  computer  will 
ask  if  you  wish  to  change  all  of  the  ratings  between  the  new  attribute  and 
each  medium  on  the  date  base.  If  you  merely  changed  the  spelling  or  wording 
of  the  original  attribute,  then  tnis  should  not  be  necessary.  If  you  have 

replaced  the  old  attribute  with  an  entirely  new  attribute,  you  may  want  to 

change  all  of  the  ratings.  To  do  so,  type  in  a  Y  <RETURN>  and  the  computer 
will  step  through  each  medium.  Determining  the  rating  for  each 
meoi um/attri bute  pair  should  be  perfonned  off-line,  as  discussed  earlier. 

You  will  notice  that  there  is  no  user-oriented  capability  for 

eliminating  (or  deleting)  a r,  attribute  from  the  media  pool  data  base.  The 

system  was  purposely  designed  without  this  function  as  a  safeguard  to  ensure 
that  no  major  disasters  occurred  in  tne  selection  process.  If  it  were 
oossible  for  a  user  to  delete  an  attribute,  the  following  problem  could 
occur.  If  a  given  user  decided  to  eliminate,  for  instance,  attribute  10, 
attribute  12  would  become  attribute  11,  and  so  forth.  Another  user,  unaware 
of  tne  change,  might  key  in  instruct- oral  attributes  from  worksheets  based  on 
the  old  numbering  sysf  n.  The  resulting  media  selections  would  tnen  be 
tota.ly  false  without  tne  present  user  necessarily  knowing  why.  So,  as  a 
safeguard,  users  can  add  or  change  atl'ibutes,  but  all  of  the  original 
attributes  will  retain  their  original  numbers.  Considering  that  the  AIMS 
system  allows  up  to  99  attributes  and  many  of  the  attributes  may  be  changed 
oy  the  user,  the  lack  of  a  capability  tc  delete  attributes  seems  a  reasonable 
sa  fegua  rd. 
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.'SECTION  V 


CREATING.  A  .\IQoE  MEDIA  POOL 

One  of  the  basic  assumption  oT'  the  AIMS  system  is  that  there  is  no 
universal  media  pool  suitable  tr-e  requirements  of  all  instructional 
situations.  Media  pools  will  rei-nsa  ;y  vary  in  both  the  media  and  the 
instructional  characteristics  \  . contain,  tne  level  of  specificity  they 
require,  and  many  other  qualities.  because  of  this,  the  AIMS  system  allows 
the  user  to  develop  project-spec i ■  :  media  pools.  The  media  pool  can  be 
edited  or  added  to  without  the  nc  e  vcr  a  programmer  to  make  the  changes. 
The  user  can  update  media  poo':.  .  selection  procedures,  and  experiment 

with  various  media  pools  with  a  o*  effort. 

STEP  1:  DEVELOPING  A  MEDIA  POCv  .  '  .  .  V. 

It  is  often  difficult  vo  ..  4- ore  to  begin  when  developing  a  unique 

media  pool.  The  following  n,  iv-  been  identified  to  provide  some 
structure  for  the  process : 

1.  Determine  the  media  c,  oerec 

The  first  task  is  to  r.Cum  :.  *  media  might  be  relevant  to  a  given 
training  project.  Make  a  :  .1  of  the  media,  or  combinations  of 

media,  to  be  included  <  o  It  is  advisable  to  allow  for  a  wide 

range  of  alternatives  c-rc-  the  oool  can  be  narrowed  if  a  shorter 
version  is  desired  for  a  qi  vt;i  ect .  Figure  A-8  is  a  sample  listing 
of  a  new  media  pool  Tibe  t  ....  It  is  a  broad  list  including  both 

cognitive  and  hands-on  typ<.  .Torn.  Once  you  have  determined  the  media 
relevant  to  your  training  nts ,  you  can  begin  the  next  task. 

2.  Determine  tne  instruct;-  eristics 

Defining  the  instruct  -  one, '  o . steri  sties  or  attributes  can  be  a 

difficult  task.  Figure  ■-  •  •■res  a  sample  list  of  attributes  from 

the  media  pool  POCl.  no.  ■  attributes  are  clusterea  into  groups 

of  similar  chariot.  f  o'. ay  Characteristics,  (2)  Response 

Mode,  (3)  Evaluation  :V,Ck:,  eedback,  and  (5)  Special  Requirements. 

Breaking  the  attributes  i  '* :  ■  ■  categories  like  this  simplifies  the 

task  of  creatine  the  me<r.->  ;na  helps  rake  the  list  itself  a  bit 

more  legible. 

Using  this  sample  as  gu  .  •■.  a  list  of  the  attributes  that  eacn 

medium  on  your  list  has.  -  \.erms  of  the  following  categories: 

st  i  cs :  Does  it  have  printed  text, 
visual  motion?  Is  it  2-dimensi ona  1 
’*  move  itself?  What  are  the  stimulus 


a.  What  are  the  disp'  :  ■ 
diagrams,  color?  bo. 
or  3-dimensional?  Dor- 
characteristics  of  the  m. 
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Ml  Cl  A  Os  i  it  FILE  :  pool 


1  '  ■ 

2.  Inij"  Access  Slides 

3.  Motion  Picture/  Videotape 

A.  O'  .  1  A„  :i  i  o 

E  .  A  „  :  :  /  T  a  p  e 

t .  f.  !  ides 
7 .  Lecture 
6 .  ;j e •n o n strati  on 

S  .  3  ■-  o  u  j  Discussion 

it .  ft  j  r  K  t uO  k 

11.  Teaching  Machine 

12.  GP.  Equi^./OP.  Environment 
1  3  .  G .  c  q  u  ;  o  ./Lab 

14.  ~eim  Trainer 

15.  Drocedural  Trainer 

16.  Part  Task  Trainer 

17.  3-0  Mock-up 

1 .  2-3  Mock-up 

15.  T  I  G  C  I  T  +  Videodisc 

20.  PLA'O  +  Videodisc 

21.  GETS 

22.  Besseler  Cue-See 


Figurf-  A-;-..  Sample  listing  of  media 
on  the  new  file  POOL. 
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d.  Wnat  '.5  toe  reoocr.se  rode  ?  Do  learners  write  an  answer 
■;  verbal  /text)?  DoThVy  speak-  tr.e  answer?  Do  they  respond  IDS  or  NG 
on  a  machine  (decision  inomcator }?  Do  they  manipulate  a  device? 
What  are  r.r response  character  sties  of  the  medium? 

c.  How  ooes  tne  student  recei vt  feedback  (evaluation)?  Is  feedbacK 

immediate?  Delayed?1  Ar?  “only  "errors  fed-back,  or  correct 

responses,  too?  is  e-ich  n-sponse  evaluated  or  is  evaluation  global, 
as  in  post-session  feeooacx  where  overall  behavior  is  evaluated? 

d.  what  are  the  environmental  condi tl ons?  Classroom?  Simulator? 

Real -worl g? 

e.  What  are  the  special  aft  L  ves?  Does  the  device  train  entire 

teams?  Does  the"-devTce  move?  Fly?  What  is  unique  about  this 
meai urn? 

f .  Is  tne  medium  ;>e-  c.  suited  for  a  certain  type  of  learning?  Cognitive 
or  general  knew?  edge?  Affective  or  attfcudinaT  learning? 
3sychomotor  or  performing  casks’ 

5.  Is  the  men- cm  Biased  towa  ra  a  certain  level  of  lea  mi ng?  Rea' 

aircraft  fee  it  ancwd  rr.ers?  SimuTaTors  for  intermediate 

learners?  Text,  cox.,,  for  oegi  oni  .19  learners* 

no  At ,  organize  .  cr  t  >.•  ;t  ..•••L  ..-,es  for  eacn  mecium  within  these 
categories  »nd  sac.  more  calono-ies  if  necessa .  Do  not  repeat 
redundanci  n  the  attributes  list. 

i.  Cross-reference  red'  -  and  tne  r  attributes 

-t  th:  s  point,  ,  ■■  uo.-  w  .  -yh  icr  s'ows  a  sample  completed  meo.: 

tool.  Note  now  tne  media  ore  li  sup.  down  tne  left-hand  column,  and  tne 

1 ‘.tributes  are  listed  acru.-,-  tne  top  rew,  creating  a  media -by -attri butt s 
matrix.  It  is  strongly  id.isec  that  you  follow  a  similar  procedure  in 
cheating  a  media  cn.f  s’ice  this  vr  1 1  reduce  the  orooability  of  making 
rro-s  an;  facilitate  rv-. . a  entry  during  the  next  step.  Therefore, 
create  a  cat-ix  with  attrir  iy  .•  listed  across  the  top  and  media  listed 
sown  the  left-hand  u1  -,:r 

ou  now  want  to  w  ;  y :  hv?  ,101  ?  it  /  cf  each  medium  to  deliver  e,.ch 

1  nstructiona'  chjrjc  rise  :  The  weighting  is  made  with  a  single 

njr.oer,  between  0  a  no  A.  it'  the  attribute  has  no  relationship  to  this 
medium,  give  it  '■ .  If  In  attribute  is  its  strong  point,  give  it  a 
If  •  t  has  tn.„  huo  ,  ■,*■/*-  a.  a  rating  between  1  and  4,  depending 

on  now  well  it.  px-ides  the  atenbute  being  considered.  In  Figure  A-1G, 
note  that  the  medium  audio-tn is  ranged  5  under  the  attribute  audio¬ 
voice,  indicating  too  st-arg  rvlati onshi p  between  the  two.  Of  course, 
workbook  is  rmx*. a  G  under  the  -.t:.-i o.rte  audio-voice  since  it  has  no 
capability  for  o>. '  ve  m  0  ;  iris  -.a  -ucteristic.  Also  notice  in  Figure  A- 
10 .that  attributes  arc  lists;  across  the  top  of  the  table  and  each 
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tribute  is  numbered.  Headings  for  each  attribute  category  are 

,  :•  eded  by  an  asterisk  (*)  and  the  columns  below  each  heading  are 

1 ; nea  out . 

STEP  2:  CREATING  A  MEDIA  POOL  DATA  BASE  (ON-LINE) 

Once  you  have  created  a  satisfactory  media  pool,  you  need  to  enter  it 
into  the  computer  as  a  data  base.  This  requi  res  getting  the  media  selection 
system  up  and  running  on  the  computer  as  discussed  in  Section  II  of  the 
manual.  When  the  system  is  ready  to  go,  the  initial  introduction  will  be 
written  on  the  CRT.  The  START  menu  offers  the  following  functions.  .  . 

1.  SELECT  MEDIA 

2.  CREATE  A  MEDIA  POOL  DATA  BASE 

3.  EDIT  A  MEDIA  POOL 

4.  PRINT  MEDIA  POOL/SELECTION  INFORMATION 

5.  RETURN  TO  OPERATING  SYSTEM 

ENTER  SELECTION? 

To  enter  your  unique  pool  into  the  computer,  type  in  a  2  <RETURN>.  The 

next  message  requests  that  you  make  up  a  file  name  for  the  media  pool.  .  . 

WHAT  IS  THE  MEDIA  POOL  FILE  NAME  TO  BE? 

The  file  name  must  be  no  more  than  eight  characters  in  length  and  may 
not  contain  spaces  or  punctuation.  It  is  advisable  that  you  select  a  file 

name  directly  relevant  to  the  media  pool,  and  less  than  eight  characters  in 

length.  This  way,  future  revisions  to  the  media  pool  can  be  numbered 
accordingly.  For  example,  AIMS2,  LESS0N13,  and  MNGMNT  are  all  valid  file 
names.  The  names  themselves  help  keep  track  of  the  training  programs  to  which 
the  media  pools  apply.  Once  you  have  successfully  entered  a  file  name,  you 
need  to  go  through  the  three-task  process  of  listing  media,  listing 
attributes,  and  cross-referencing  media  and  attributes,  but  this  time  you 
will  be  doing  it  on-line. 

After  the  file  name  has  been  entered,  the  computer  requests.  .  . 

TASK  1 

HOW  MANY  MEDIA  WILL  BE  CONTAINED  IN  THIS  POOL? 

Type  in  the  number  on  the  worksheet  followed  by  <RETURN>.  The  computer 
then  displays.  .  . 

TYPE  IN  EACH  MEDIUM  NAME  FOLLOWED  BY  A  <RETURN> 

You  want  to  type  in  the  first  medium  name,  enter  it  (by  pressing 
<RETURN>) ,  type  in  the  second  medium  name,  enter  it,  type  in  the  third  medium 
name,  enter  it,  etc.  The  computer  will  number  them  for  you  as  you  go  along, 
so  you  can  keep  checking  back  to  the  worksheet  to  make  sure  you  have  your 
numbering  right.  If  you  make  a  mistake,  type  in  a  999  and  the  computer  will 
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help  you  get  things  straightened  out.  When  you  have  typed  in  the  last 
medium,  the  CRT  will  automatically  display  the  next  message.  .  . 

TASK  2 

T JPE  IN  THE  ATTRIBUTES  AND  HEADINGS,  FOLLOWING  EACH  ONE  WITH 

A  'RETURN*.  .  . 

ENTER  AN  ATTRIBUTE  OR  A  *HEADING  (1)? 

(OR  "0"  WHEN  DONE,  999  FOR  HELP) 

At  this  point,  make  sure  you  have  all  of  the  headings  clearly  marked  on 
the  worksheet  with  asterisks,  "*".  teke  sure  your  attributes  are  numbered  1 
through  n,  and  that  any  headings  or  comments  are  not  numbered.  This  is 
important,  since  anything  that  is  not  led  with  a  will  be  numbered  and 
considered  an  attribute  by  the  computer. 

In  response  to  the  CRT  request,  type  in  an  attribute  or  a  ^heading.  An 
attribute  will  receive  the  numerical  value  that  appears  in  parentheses  on  the 
CRT.  A  ^heading  will  receive  no  numerical  value.  Keep  track  of  the  numbers 
as  they  progress,  and  make  sure  they  agree  with  the  numbers  on  the  worksheet. 
If  you  make  an  error,  enter  a  999  as  an  attribute,  and  the  computer  will  help 
you  correct  things.  When  you  have  typed  in  the  last  attribute,  type  in  a  0 
<RETURN>  and  you  will  get  the  next  message.  .  . 

TASK  3 

CROSS-REFERENCE  MEDIA  AND  ATTRIBUTES 

This  task  results  in  the  most  errors  because  the  user  becomes  careless 
in  reading  vrfiat  is  on  the  screen.  Basically,  you  want  to  put  in  all  of  the 
rating  numbers  that  relate  the  media  to  their  attributes.  When  you  are  ready 
to  proceed,  the  computer  informs  you  of  the  medium  you  will  be  rating,  then 
requests  that  you  rate  each  relationship.  It  begins  with  the  first  medium 
and  the  fist  attribute,  proceeds  to  the  second  attribute,  the  third 
attribute,  etc.,  all  the  way  to  the  last  medium  and  the  last  attribute.  Each 
request  will  appear  as  a  small  model  of  the  worksheet  on  the  CRT. 
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SECTION  VI 


REFINING  THE  SELECTION  PROCESS 

While  performing  media  selections  with  the  standardized  AIMS  media  pool 
(or  even  a  unique  media  pool),  you  may  find  that  the  selections  are  adequate 
in  a  general  sense,  but  could  be  improved  to  fit  the  training  program  at 
hand.  This  section  provides  examples  of  how  to  take  information  from 
preliminary  media  selections  to  refine  the  selection  process  by  using  the 
AIMS  edit  functions. 

For  example,  perhaps  you  have  a  medium  or  several  media  in-house  which 
are  not  on  the  AIMS  media  pool.  You  can  rate  each  new  medium  with  the 
attributes  on  the  AIMS  media  pool,  as  described  in  Section  V  of  this  manual. 
Then,  using  the  AIMS  edit  function,  ADD  A  MEDIUM,  you  can  add  the  new  medium 
or  media  to  the  existing  AIMS  media  pool  without  disturbing  the  rest  of  the 
data  base. 

In  another  example,  assume  that  you  have  added  some  media,  changed 
others,  and  performed  several  media  selections  for  a  group  of  objectives.  By 
analyzing  the  selections,  you  may  determine  that  the  attributes  on  the 
present  media  pool  do  not  sufficiently  discriminate  among  the  available 
media.  Of  course,  determining  how  to  better  discriminate  among  media  is  a 
judgmental  task  that  requires  a  certain  amount  of  knowledge  or  experience 
from  the  user. 

Assume  the  computer  is  generally  selecting  the  following  media  for  a 
group  of  objectives: 

1.  WORKBOOK 

2.  LECTURE 

3.  SLIDE /TAPE 

4.  RANDOM  ACCESS  SLIDES 

5.  CAI 

6.  BESSLER  CUE -SEE 

Now,  suppose  you  know  that  CAI  and  random  access  slides  should  not  be 
selected  for  these  objectives  because  some  of  the  students  are  in  an  early 
stage  of  learning  and  CAI  and  random  access  slides  are  only  used  for 
intermediate  level  students  at  the  training  site.  It  becomes  clear  that  the 
AIMS  system  needs  to  make  finer  discriminations  among  beginning- level  and 
intermediate-level  students.  This  translates  to  the  addition  of  two  new 
attributes  to  the  present  media  pool  data  base.  Specifically: 

1.  EARLY  STAGE  OF  LEARNING 

2.  INTERMEDIATE  STAGE  OF  LEARNING 

Then  the  user's  job  is  to  rate  each  medium  in  the  media  pool  on  a  0  (no 
relationship)  to  5  (strong  relationship)  scale  for  these  two  new  attributes. 
The  media  CAI  and  random  access  slides  would  receive  a  0  rating  on  the  EARLY 
STAGE  OF  LEARNING  attribute,  since  these  media  are  used  solely  for 
intermediate-level  students.  Suppose  you  also  know  that  workbooks  and 
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slide/tape  are  used  strictly  for  beginning- level  students.  These  two  media 
would  receive  a  0  rating  on  the  INTERMEDIATE  LEVEL  OF  LEARNING  attribute,  and 
all  other  media  would  receive  some  rating  greater  than  0  for  this  attribute. 

In  the  next  step  you  would  go  to  the  computer,  run  the  AIMS  system,  and 
from  there  call  up  the  edit  function,  ADD  AN  ATTRIBUTE.  Using  the 
Instructions  provided,  you  would  add  the  attribute  EARLY  STAGE  OF  LEARNING 
and  rate  each  medium  for  this  new  attribute,  being  sure  to  assign  0  ratings 
to  the  media  CAI  and  random  access  slides.  Next  you  would  add  the  attribute 
INTERMEDIATE  LEVEL  OF  LEARNING  and  rank  each  medium  accordingly.  Of  course, 
workbook  and  slide/tape  would  receive  a  0  rating  since  they  are  not  used  at 
this  stage  of  learning. 

Once  the  new  attributes  have  been  added  to  the  media  pool  data  base,  you 
would  use  the  print  functions  on  the  AIMS  system  to  produce  a  new  selection 
worksheet.  The  new  selection  worksheet  would  include  the  new  attributes  and 
the  usual  check-off  boxes  for  choosing  the  attributes  as  characteristics  of 
the  objective  at  hand.  Having  these  additional  attributes  on  the  data  base 
and  worksheets  would  have  the  following  effects  {assuming  the  media  were 
rated  as  they  were  in  this  example): 

1.  For  those  objectives  in  which  EARLY  STAGE  OF  LEARNING  is  determined 
to  be  an  instructional  characteristic  on  the  worksheet,  the  computer 
will  select  all  of  the  appropriate  media  except  the  0-rated  media 
CAI  and  random  access  slides.  All  other  media  will  be  selected  and 
rank -ordered  in  the  usual  fashion. 

2.  For  those  objectives  in  which  INTERMEDIATE  LEVEL  OF  LEARNING  is 
determined  to  be  an  Instructional  characteristic  on  the  worksheet, 
the  computer  will  select  all  of  the  appropriate  media  except  the  0- 
rated  media  workbook  and  slide/tape.  All  other  media  will  be 
selected  and  rank-ordered  in  the  usual  fashion. 

3.  For  those  objectives  in  which  neither  of  the  new  attributes  (EARLY 
STAGE  OF  LEARNING  or  INTERMEDIAmTA’GE  OF  LEARNING)  were  selected 
as  instructional  characteristics,  the  computer  will  consider  all  of 
the  media  in  the  pool  for  selection.  Including  workbook,  slide/tape, 
random  access  slides  and  CAI.  In  other  words.  If  neither  of  these 
attributes  Is  considered  to  be  of  Importance  to  the  objective  being 
"mediated,"  then  the  computer  will  make  no  discriminations  based  on 
these  attributes  when  selecting  media. 

By  adding  two  new  attributes,  you  have  provided  the  option  of  refining 
the  selection  process.  If  you  choose  to  define  an  objective  as  being  in 
either  the  early  or  intermediate  stage  of  learning,  you  will  be  eliminating 
certain  media  from  consideration  for  selection.  Specifically,  those  media 
which  were  rated  as  having  a  0  relationship  with  the  determined  attribute. 
If  you  choose  not  to  specify  a  level  of  learning  for  a  given  objective,  then 
neither  of  these  attributes  will  have  an  effect  on  the  computer's  selection 
of  media  and  all  of  the  media  In  the  media  pool  will  be  considered  for 
selection  and  rank-ordered  appropriately. 
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In  yet  another  example,  using  the  standardized  AIMS  media  pool  as  a 
data  base,  assume  that  you  find  the  computer  often  selecting  the  following 
two  media  for  certain  objectives: 

1.  PART  TASK  TRAINER 

2.  PROCEDURAL  TRAINER 

As  the  user,  assume  that  you  know  that  the  main  difference  between  the 
part  task  and  procedural  trainers  at  your  site  is  that  the  part  task  trainer 
provides  a  computer-operated  view  of  a  moving  target  on  a  radar  screen,  vfcile 
the  procedural  trainer  does  not.  This  fact  makes  MOVING  TARGET  an  important 
attribute  for  selecting  media.  As  in  the  last  example,  you  could  simply  add 
MOVING  TARGET  as  an  attribute  on  the  media  pool  data  base  to  further  refine 
the  computer's  selections.  But  in  this  instance,  assume  that  the  attribute 
COLOR  on  the  data  base  is  never  used.  B>  using  the  AIMS  edit  function, 
CHANGE  AN  ATTRIBUTE,  you  could  replace  the  attribute  COLOR  with  the  attribute 
MOVING  TARGET.  Then  you  could  re-rate  all  of  the  media  with  this  new 
attribute,  and  thereby  refine  the  selection  procedure.  The  effect  is  the 
same  as  in  the  previous  example  of  adding  attributes,  except  you  have 
replaced  an  irrelevant  attribute  with  a  relevant  one. 

Of  course,  experience  in  instructional  design  and  development,  as  well 
as  experience  with  the  AIMS  system,  will  provide  the  best  lessons  for 
refining  the  selection  process.  However,  as  a  general  procedure  for  refining 
AIMS  media  selections,  it  is  recommended  that  you  follow  these  steps: 

1.  Try  some  preliminary  media  selections  and  analyze  the  selections 
made. 

2.  Determine  which  media  are  relevant  to  the  training  program  at  hand 
and  not  in  the  present  media  pool  data  base,  then  add  them  to 
the  media  pool  using  the  ADD  A  MEDIUM  edit  function. 

3.  Consider  the  possibilities  for  further  discriminating  among  media 
selections,  and  translate  these  possibilities  into  concrete 
instructional  characteristics  or  media  attributes. 

4.  Determine  If  any  media  attributes  or  instructional  characteristics 
on  the  present  media  pool  data  base  are  Irrelevant  to  the  needs  of 
the  training  program  at  hand,  then  replace  them  with  relevant 
attributes  or  instructional  characteristics  using  the  CHANGE  AN 
ATTRIBUTE  edit  function. 

5.  Add  any  media  attributes  or  instructional  characteristics  that  are 
relevant  to  the  training  program  at  hand  using  the  ADD  AN  ATTRIBUTE 
edit  function. 
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